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1.  INTRODUCTION 


This  is  the  fifth  semi-annual  report  of  the  project  entitled  "Automated  Handling  and  Assembling 
of  Non-Rigid  Objects".  This  report  covers  the  period  from  January  16,  1992  to  July  15,  1992. 

The  objective  of  this  research  project  is  to  identify  mechanisms  for  automated  handling  and 
assembling  of  non-rigid  objects.  To  accomplish  the  goal,  four  aspects  of  the  problem  have  been 
identified  in  our  original  proposal.  They  are  - 

a.  Study  the  deformation  characteristics  of  non-rigid  materials. 

b.  Investigate  optimal  tool  structures  for  handling  non-rigid  objects. 

c.  Develop  sensing  mechanisms  to  locate  non-rigid  objects. 

d.  Develop  motion  control  mechanisms  for  the  host  machines. 

To  explore  these  aspects,  many  technical  issues  need  to  be  examined.  During  the  past  six 
months,  two  topics  were  studied  by  the  Principal  Investigator  and  his  team.  The  first  topic 
investigates  methods  for  detecting  smooth  surfaces  of  deformed  plates.  This  study  is  beneficial  in 
many  applications  where  metal  plates  are  used  such  as  in  shipbuilding,  automobile,  and  aerospace 
industries.  In  order  to  automatically  handle  the  deformable  plates,  the  shape  of  the  plate  must  be 
detected.  One  difficulty  in  detecting  such  a  shape  is  that  the  smooth  surfaces  of  deformed  plates  do 
not  have  matching  features  for  stereo  vision  to  use.  We  propose  a  passive-active  stereo  vision 
method  for  this  purpose.  Passive  stereo  vision  without  the  projection  of  structured  light  is  used  to 
obtain  the  boundary  of  the  smooth  surfaces,  and  active  stereo  vision  with  the  projection  of 
structured  light  is  applied  to  detect  the  details  of  the  surface.  An  inherent  problem  in  stereo  vision, 
called  the  false  boundary  problem,  is  identified  and  solved  by  calibrating  the  structured  light. 

The  second  topic  continues  the  study  of  digitizing  an  unknown  surface  using  laser 
displacement  sensors  (LDS).  As  mentioned  in  the  fourth  semi-annual  report,  vision  sensors  can 
only  identify  the  global  shape  of  a  deformed  object.  To  precisely  measure  the  profile  of  an 
unknown  surfare,  a  touch  probe  has  customarily  been  used  to  digitize  the  surface.  This  process  of 
digitizing  is  very  slow  due  to  a  large  number  of  contacts  that  need  to  be  made  by  the  probe.  We 
have  proposed  to  use  non-contact  laser  displacement  sensors  (LDS)  to  speed  up  the  measurement 
process.  In  the  fourth  semi-annual  report,  we  described  two  approaches  to  using  the  LDS,  i.e., 
the  multiple  beam  approach  and  the  extrapolation  approach.  During  the  past  six  months,  we  also 
studied  precision  issues  associated  with  the  two  approaches.  Through  this  study,  we  have  shown 
that  the  digitizing  error  parameters  are  bounded  functions  when  using  these  two  approaches  to 
digitize  an  unknown  surface  which  has  a  finite  maximum  curvature.  These  functions  can  be  used 
to  determine  an  adequate  sampling  arc  length,  if  the  maximum  surface  curvature  is  know  n,  or  they 
can  be  used  to  determine  the  maximum  surface  curvature  that  can  be  accurately  digitized  for  a  given 
sampling  length. 
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In  the  next  two  sections,  research  results  of  the  above  two  topics  will  be  summarized. 
Detailed  descriptions  of  these  two  studies  can  be  found  in  Appendices  A  and  B  respectively. 

2.  PASSIVE  AND  ACTIVE  STEREO  VISION  FOR  SMOOTH  SURFACE 
DETECTION  OF  DEFORMED  PLATES 

In  this  study,  we  consider  to  use  stereo  vision  to  detect  the  3-D  shape  of  deformed  plates. 
This  study  is  necessary  when  the  handling  of  deformable  plates  needs  to  be  automated. 

Stereo  vision  is  an  attractive  technique  for  depth  perception  in  many  applications.  For 
detecting  the  shape  of  deformed  plates,  however,  special  problems  need  to  be  considered.  The 
first  problem  is  called  the  false  boundary  problem,  which  is  caused  by  the  curved  surface  of 
deformed  plates.  When  this  occurs,  the  actual  boundary  of  the  surface  may  be  hidden  from  the 
camera.  The  second  problem  is  that  the  smooth  surface  does  not  generate  matching  features  in 
the  image  pair  of  stereo  vision.  As  a  result,  the  depth  of  the  object  cannot  be  derived  directly 
by  matching  extracted  features  through  the  triangulation  method.  To  solve  the  preceding 
problems,  we  propose  a  passive-active  stereo  vision  method  plus  a  dynamic  sensing  process. 
Essentially,  we  use  passive  stereo  vision  to  detect  the  boundary  of  the  curved  surface.  Active 
stereo  vision  with  the  projection  of  structured  light  is  then  used  to  detect  the  details  of  the 
surface.  The  passive-active  stereo  vision  is  further  enhanced  by  adynamic  sensing  process. 
That  is,  the  stereo  vision  system  is  dynamically  positioned  and  oriented  to  effectively  detect  the 
shape  of  a  deformed  plate. 

2.1  Passive  Stereo  Vision  to  Reduce  the  Likelihood  of  the  FaIse>Boundary 
Problem 

When  a  surface  is  seen  by  the  two  cameras  of  stereo  vision,  the  boundary  of  the  surface 
detected  in  each  camera  might  not  correspond  to  the  actual  boundary  of  the  surface.  Shown  in 
Fig.  1  are  two  examples  of  this  problem.  In  (a)  of  Fig.  1,  the  right  side  of  the  surface 
boundary  is  hidden  from  the  left  camera,  and  the  left  side  of  the  surface  boundary  is  hidden 
from  the  right  camera.  In  (b)  of  Fig.  1,  the  lower  side  of  the  surface  boundary  is  hidden  from 
both  cameras. 

Two  methods  are  proposed  to  eliminate  the  false  boundary  problem.  One  method  is  to 
orient  the  center  line  of  the  stereo  vision  sensor  in  order  that  the  center  line  will  bisect  and  be 
perpendicular  to  the  line  segment  connecting  the  two  end  points  of  the  most  bent  curve 
drawn  on  the  surface.  Such  an  arrangement  will  reduce  the  likelihood  of  the  surface  boundary 
been  hidden  from  the  views  of  the  cameras.  The  other  method  is  to  move  the  stereo  vision 
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Fig.  1  Two  examples  of  the  false-boundary  problem  in  stereo  vision 

sensor  farther  away  from  the  surface.  In  this  way,  the  view  angle  of  the  camera  formed  by  the 
curved  surface  will  decrease  and  may  eliminate  the  false  boundary  problem.  These  two 
approaches  are  illustrated  in  Fig.  2  and  Fig.  3  respectively. 

The  above  two  approaches  can  reduce  the  likelihood  of  the  false  boundary  problem,  but 
cannot  eliminate  the  false  boundary  problem.  The  passive  stereo  vision  method  mast  cooperate 
with  active  stereo  vision  and  dynamic  sensing  to  totally  eliminate  the  false  boundary  problem. 

2.2  Matching  Criteria  in  Active  Stereo  Vision 

Since  the  surface  of  a  deformed  plate  is  smooth,  it  is  necessary  to  use  active  vision  to  detect  the 
3-D  shape  of  the  surface.  For  active  stereo  vision,  structured  light  with  certain  patterns  must 
be  used.  We  choose  to  use  an  array  of  squares  as  shown  in  Fig.  4  to  be  the  light  pattern.  The 
comers  of  the  squares  are  used  as  matching  point  for  deriving  the  data  points  in  the  3-D  space. 
Squares  are  chosen  because  they  are  simple  and  provide  more  matching  points  than  other 
simple  features  such  as  circles  and  triangles. 
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Fig.  2:  (a)  The  false  boundary  problem  occurs. 

(b)  The  center  line  of  the  stereo  vision  sensor  is  bisecting  and  perpendicular  to 
the  line  segment  to  reduce  the  likelihood  of  the  false- boundary  problem. 


The  view  of  the  The  vie**  of  the 
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moving  far  away  moving  far  sway 

frotn  tie  surface  from  the  surface 


<*> 

Fig.  3:  (a)  The  false-boundary  problem  occurs,  (b)  The  vision  sensor  is  moved  farther 

away  from  the  surface  to  reduce  the  likelihood  of  the  false  boundary  problem. 
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The  key  problem  in  active  stereo  vision  is  the  matching  of  the  squares  in  the  image  pair. 
The  matching  criteria  that  we  use  can  be  explained  by  Fig.  5.  An  arbitrary  surface  is  assumed 
to  be  flat  and  parallel  to  the  image  planes  of  the  cameras  (imaginary  flat  surface  in  Fig.  5).  An 
imaginary  plane,  named  "structured  light  plane",  is  defined  as  the  intersection  of  the  structured 
light  and  an  arbitrary  plane  perpendicular  to  the  center  ray  of  the  structured  light.  The  center 
ray  of  the  structured  light  is  oriented  parallel  to  the  optical  axes  of  the  cameras  while  the 
structured  light  is  oriented  parallel  to  the  optical  axes  of  the  cameras.  The  created  matching 
squares  on  the  structured  light  plane  will  then  lie  parallel  to  the  x  and  y  axes  of  the  structured 
light  plane. 

The  uniqueness  constraint  and  continuity  constraint  identified  by  Marr  and  Poggio  [1]  can 
be  used  for  matching  the  squares  in  the  case  defined  above.  According  to  the  uniqueness 
constraint,  there  will  be  only  one  square  in  the  images  of  the  cameras  corresponding  to  one 
square  on  the  flat  surface.  According  to  the  continuity  constraint,  adjacent  squares  in  the  image 
will  be  projected  from  adjacent  squares  on  the  flat  surface.  With  the  use  of  uniqueness  and 
continuity  constraints,  each  square  in  the  horizontal  array  in  the  image  pair  can  be  matched 
according  to  their  relative  positions  in  the  x  direction.  The  above  description  defines  the 
matching  criteria. 


Fig.  4  The  patterns  of  the  structured  light  in  active  stereo  vision 
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Fig.  5  The  explanation  of  the  matching  criteria  in  active  stereo  vision 

In  reality,  a  deformed  plate  may  have  a  curved  surface.  As  a  result,  the  positions  of  the 
squares  in  the  image  pair  will  be  shifted  from  their  positions  projected  in  the  flat  surface  case. 
This,  however,  will  not  cause  any  problems  in  feature  matching.  The  space  between  two 
adjacent  squares  in  the  x  direction  can  be  adjusted  to  be  sufficiently  small  such  that  two 
adjacent  squares  in  the  structured  light  plane  will  be  projected  to  the  image  pairs  as  two  adjacent 
squares  in  the  x  direction.  Furthermore,  the  space  between  two  adjacent  horizontal  square 
arrays  in  the  projection  plane  can  be  designed  sufficiently  large  such  that  two  squares  from  two 
adjacent  horizontal  square  arrays  will  never  be  projected  as  two  adjacent  squares  in  the  image 
pairs.  The  above  two  spacing  criteria  are  further  explained  in  Fig.  6.  Apparently,  the  space 
between  the  adjacent  squares  must  be  selected  in  accordance  with  the  maximum  curvature  of 
the  deformed  plates. 

2.9  Dynamic  Sensing  Process 

Based  on  the  passive-active  stereo  vision  mechanism  discussed  above,  the  dynamic  sensing 
process  for  detecting  the  shape  of  the  curved/smooth  surface  include  three  stages. 

In  the  first  stage,  passive  stereo  vision  is  applied  to  obtain  a  rough  boundary  of  the  surface. 
The  detected  boundary  may  not  be  the  real  boundary  of  the  surface  because  of  the  false 
boundary  problem.  The  rough  boundary  detected  in  this  stage  will  define  a  boundary  plane. 
The  boundary  plane  is  defined  to  be  the  best-fit  plane  of  the  detected  boundary.  That  is,  the 
overall  distance  from  the  detected  boundary  to  the  boundary  plane  is  at  minimum.  The 
boundary  plane  will  guide  the  movement  of  the  cameras  in  the  second  stage. 
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:  overlapping  horizontal  lines 


(a)  (b) 


Fig.  6  In  (a)  and  (b),  squares  A  and  B  are  projected  from  the  same  horizontal 
square  array,  and  C  and  D  are  projected  from  another  horizontal  square 
array.  A  and  D  should  not  be  interpreted  as  on  the  same  horizontal  array 
and  should  be  avoided. 

In  the  second  stage  of  the  sensing  process,  the  cameras  will  be  moved  to  a  closer  position 
to  the  surface  so  that  the  distance  between  the  cameras  and  the  surface  will  be  within  the 
effective  focal  length  of  the  cameras.  In  addition,  the  image  planes  are  oriented  parallel  to  the 
boundary  plane  found  in  the  first  stage.  The  boundary  detected  by  passive  vision  in  this  stage 
reduces  the  likelihood  of  the  false  boundary  problem  and  will  define  the  area  where  active 
stereo  vision  should  be  applied. 

In  the  third  stage  of  the  sensing  process,  without  moving  the  cameras,  the  projector  moves 
to  a  proper  position  and  orientation  to  project  the  structured  light  on  the  surface.  The  images  of 
the  surface  are  acquired  by  the  two  cameras.  The  created  matching  squares  within  the  surface 
boundary  in  the  image  can  be  extracted.  The  existence  of  the  false  boundary  problem  can  be 
detected  by  comparing  the  number  of  squares  extracted  in  the  image  pairs  to  the  actual  number 
of  squares  within  the  square  boundary.  If  the  two  numbers  are  different,  the  false  boundary 
problem  is  detected.  The  cameras  should  be  moved  farther  away  from  the  surface  to  see  the 
real  boundary. 
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After  the  false  boundary  problem  is  totally  avoided,  the  comers  of  the  squares  in  the  image 
pair  can  be  used  as  the  matching  points  for  deriving  the  data  point  of  the  surface. 

The  sensing  process  as  described  above  has  been  confirmed  by  experiments  done  in  our 
laboratory.  A  PUMA  robot  was  used  to  carry  the  stereo  vision  system,  and  another  PUMA 
robot  was  used  to  carry  the  projector.  The  results  of  the  experiments  proves  that  the  proposed 
passive-active  stereo  vision  is  effective  for  detecting  the  shape  of  the  smooth  surface  of 
deformed  plates. 

3.  ERROR  ANALYSIS  ON  THE  NON-CONTACT  3-D  DIGITIZING  SYSTEM 

In  recent  years,  computer  controlled  digitizing  systems  such  as  coordinated  measuring 
machines  (CMM)  have  become  very  popular  for  measuring  a  complex  sculptured  surface. 
Traditional  digitizing  systems  use  a  touch  probe  to  measure  individual  points  on  the  surface.  This 
is  a  slow  process  since  the  probe  has  to  make  contact  with  the  surface  many  times.  We  have 
proposed  to  use  Laser  Displacement  Sensors  (LDS)  to  replace  the  touch  probe.  The  LDS  can 
digitize  an  unknown  surface  without  any  contact.  As  a  result,  the  speed  of  measurement  can  be 
increased. 

However,  the  LDS  has  certain  limitations.  First,  the  effective  stand  off  distance  of  a  LDS  is 
limited,  and  secondly,  the  laser  beam  must  be  oriented  perpendicular  to  the  object  surface.  If  these 
two  limitations  are  violated,  either  the  LDS  fails  to  give  a  measurement  or  the  measurement  given  is 
not  precise.  Since  the  surface  is  unknown,  it  is  difficult  to  keep  the  LDS  from  violating  the  two 
limitations.  We  have  developed  an  active  sensing  concept.  The  idea  is  to  install  the  LDS  on  an 
articulate  CMM  so  that  the  position  and  orientation  of  the  LDS  can  be  actively  adjusted.  For 
adjusting  the  CMM,  we  have  developed  two  methods  to  predict  the  variation  of  the  surface.  They 
are  the  Multiple  Beam  Approach  and  the  Extrapolation  Approach.  The  concepts  of  these  two 
approaches  are  described  in  the  previous  semi-annual  report  (the  fourth  semi-annual  report).  For 
convenience,  they  are  briefly  discussed  here  again. 

The  multiple  beam  approach  uses  three  LDS  to  measure  the  curvature  and  tangent  of  an 
unknown  surface.  As  a  result,  the  LDS  can  locally  follow  surface  variations  although  the  global 
shape  of  the  surface  is  not  known.  The  n^1  order  polynomial  extrapolator,  on  the  other  hand,  is 
the  process  of  extrapolating  a  continuous  function  between  the  present  sampling  instants  and  the 
next  sampling  instant.  The  polynomial  has  the  following  standard  form: 

z(x)  =  anxn  +  an-ixnl  +  ...  +  aix  +  ao  (1) 
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The  n+1  unknown  coefficients  in  the  polynomial  are  solved  for  by  using  the  previous  n+1  data 
points.  Once  all  the  coefficients  are  solved,  the  next  sampling  point  can  be  predicated  by  using  this 
same  polynomial. 

Over  the  past  six  months,  we  have  further  evaluated  the  two  digitizing  approaches  by  analyzing 
the  error  parameters  associate  with  the  digitizing  process.  There  are  three  error  parameters 
associated  with  the  digitizing  process:  p,  0  and  As.  The  first  two  error  parameters,  p  and  0,  are  a 
result  of  limitations  associated  with  the  LDS.  p  is  the  displacement  between  the  LDS's  reference 
position  and  the  object's  surface.  0  is  the  angle  between  the  laser  beam  and  the  surface  normal 
vector  (Fig.  7). 


Fig.  7  Digitizing  error  parameters 

In  order  to  have  a  precise  measurement,  p  and  0  must  be  bounded  within  the  manufacturer's 
specifications.  The  last  parameters,  As,  is  related  to  the  sampling  arc  length.  It  is  the  difference 
between  the  desired  sampling  arc  length,  s,  and  the  actual  arc  length,  s+As,  as  measured  between 
adjacent  digitized  points.  Our  study  evaluates  the  bounds  on  the  three  parameters  as  a  result  of 
using  the  two  digitizing  approaches. 

The  bounds  on  these  error  parameters  are  developed  through  three  steps.  In  the  first  step,  we 
prove  that  the  next  sampling  point  is  within  a  bounded  area  if  the  sampling  arc  length  s  is  bounded 
and  the  maximum  curvature  of  the  surface  is  finite.  In  the  second  step,  we  prove  that  within  the 
bounded  area  defined  in  the  first  step,  any  sampling  point  will  have  a  bounded  maximum  slope  and 
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a  bounded  minimum  slope.  In  the  third  step,  based  on  the  worst  case  conditions,  we  derive  the 
error  parameters  as  functions  of  the  ratio  between  the  sampling  arc  length  and  radius  of  maximum 
curvature.  These  three  steps  are  now  presented  in  greater  detail. 

A.  A  Plane  Curve  with  Finite  Maximum  Curvature  is  Bounded  Locally 

In  the  two  digitizing  approaches,  the  surface  is  digitized  one  row  at  a  time  where  the  row's  are 
formed  by  the  intersection  of  parallel  planes  with  the  surface.  The  intersection  of  a  plane  with  the 
surface  forms  a  plane  curve.  We  have  shown  that  a  plane  curve  which  is  known  to  have  a  finite 
maximum  curvature,  Km,  and  to  pass  through  the  point  PG,  with  a  tangent  vector,  T0,  is  bounded 
locally  by  two  tangent  circles  with  radii,  R,  where  R  =  l/»cm  (Fig.  8).  A  sufficient  condition  for 
this  locality  is  Isl  £  {nil)  R  where  s  is  the  curve’s  arc  length  measured  from  the  point  PG.  The 
curve  will  not  cross  into  these  tangent  circles  as  long  as  the  arc  length  satisfies  this  condition.  The 
shaded  area  in  Fig.  8  highlights  the  locally  bounded  region  of  the  next  sampling  point . 


B.  A  Curve's  Slope  is  Bounded  in  the  Locally  Bounded  Region 

Suppose  the  following  four  things  are  known  about  a  plane  curve:  1)  It  has  a  finite  maximum 
curvature,  2)  It  passes  through  the  point  P0, 3)  It  has  a  known  tangent  vector,  T0,  at  the  point  P0, 
and  4)  It  passes  through  the  point  P2  where  7  ^  is  located  in  the  locally  bounded  region  defined 
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above.  With  these  constraints,  the  slope  of  the  curve  at  the  point  P2  is  bounded  by  the  maximum 
and  minimum  slopes  mj  and  ms  respectively  (Fig.  9). 

To  determine  the  worst  possible  orientation  that  the  plane  curve  might  have  with  the  LDS's 
beam,  maximum  and  minimum  slope  values  need  to  be  determined  for  points  along  a  line  which 
extends  across  the  locally  bounded  region.  This  line  represents  the  LED's  beam,  and  it  intersects 
the  origin  of  the  lower  tangent  circle  while  forming  an  angle,  y,  with  the  w  axis  as  shown  in  (Fig. 
10).  Clearly,  if  the  plane  curve  follows  the  top  tangent  circle,  the  angle  between  the  laser  beam  and 
the  normal  vector  of  the  plane  curve  will  be  the  largest.  The  other  error  parameters  are  also 
maximized  under  these  conditions. 

C.  Deriving  Error  Parameter  Functions 

Based  on  the  worst  prediction  conditions  as  defined  in  A  and  B,  we  derived  error  parameters 
as  functions  of  the  ratio  of  sampling  arc  length  to  radius  of  maximum  curvature.  These  functions 
are  respectively  written  as: 

+  sin  l{sin(^-)  -  sin(^-)^2cos(^)-l  },  (2) 

=2cos(^-)-  1  -  yjl cos(^H  (3) 

and 

6=^-  +  sin  -  sin(p-)‘\j2cos(^)-l  }.  (4) 

From  the  above  three  equations,  the  error  parameters  are  shown  to  be  monotonically  increasing 
functions  of  the  ratio  of  the  sampling  arc  length  to  radius  of  maximum  curvature.  By  using  these 
equations,  we  can  determine  what  the  sampling  arc  length  should  be  given  the  limitations  on  the 
sampling  errors. 

4.  RESEARCH  ACTIVITIES  FOR  THE  NEXT  PERIOD 

In  the  next  period,  we  plan  to  investigate  and  complete  the  following  two  topics: 

a.  Graduate  student  Rached  Zantout  has  been  studying  the  deformation  of  two  dimensional 
objects  since  the  beginning  of  this  year.  He  is  currently  investigating  how  to  identify  the 
deformation  characteristics  of  a  two  dimensional  plate.  This  investigation  contains  two 
steps: 
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Fig.  9  Bound  on  local  slope  values 


fig.  10  Extreme  LDS  beam  orientation 


(1)  identifying  the  deformation  of  a  plate  under  external  loads,  and  (2)  calculating  the 
parameters  which  governs  the  deformation  behavior  of  the  plates.  A  geodesic  approach  is 
being  used  to  measure  the  deformation,  and  a  Finite  Element  Method  (FEM)  is  being  used 
to  determine  the  deformation  characteristics.  We  expect  that  this  study  can  be  completed  by 
the  end  of  the  next  period. 

b.  A  new  approach  for  surface  representation  of  a  deformed  plate.  Last  year,  we  developed  a 
new  approach,  based  on  energy  minimization  and  Coon's  patches,  to  represent  the  surface 
of  a  deformed  plate.  Recently,  we  have  found  another  new  approach  which  may  be  more 
convenient  than  the  previous  approach.  This  new  approach  is  based  on  the  perturbation 
technique.  By  using  the  perturbation  technique,  the  number  of  patches  used  to  represent  a 
surface  can  be  significantly  reduced. 

The  results  of  the  above  research  activities  will  be  presented  in  the  next  semi-annual  report 
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Passive  and  Active  Stereo  Vision  For  Smooth  Surface 
Detection  of  Deformed  Plates1 
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Abstract 

Passive  stereo  vision  is  proposed  to  work  in  a  complimentary  manner  with  active 
stereo  vision  for  the  detection  of  smooth  surfaces  of  deformed  plates.  Passive  stereo 
vision  without  the  projection  of  structured  light  is  used  to  obtain  the  boundary  of 
the  smooth  surfaces;  whereas  active  stereo  vision  with  the  projection  of  structured 
light  is  applied  to  detect  the  details  of  the  surfaces.  An  inherent  problem  in  stereo 
vision,  called  false  boundary  problem,  is  identified  and  solved  by  calibrating  the 
structured  light.  The  matching  criteria  in  active  stereo  vision  and  the  sensing  process 
of  the  proposed  approach  are  presented.  An  experiment  was  conducted  to  test  the 
effectiveness  of  the  proposed  approach. 
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I.  Introduction 


Making  the  machine  “see”  an  object  is  one  of  the  most  important  tasks  in  machine 
intelligence,  manufacturing  automation,  and  robotics.  To  enable  a  machine  “see” 
involves  two  works:  sensor  detection  to  obtain  data  points  of  the  object  and  shape 
representation  of  the  object  for  the  process  of  object  recognition,  inspection,  and 
manipulation  for  the  machine.  3-D  shape  reconstruction  of  the  object  is  the  process 
of  these  two  works,  obtaining  data  points  of  3-D  objects  and  constructing  its  shape 
from  the  sensed  data  points  through  interpolation  or  approximation. 

A  special  case  in  3-D  shape  detection  using  stereo  vision  is  considered  in  this 
paper.  We  are  concerned  about  how  to  detect  the  smooth  surfaces  of  metal  plates. 
This  study  is  stimulated  by  the  observation  that  metal  plates  are  often  used  in  many 
applications.  For  example,  in  shipbuilding,  aerospace,  and  automobile  industries, 
vehicle  bodies  are  the  essential  parts  of  their  products.  These  bodies  are  usually 
constructed  by  assembling  together  numerous  pieces  of  metal  plates.  Vision  sensor  is 
chosen  for  shape  detection  of  these  parts  because  of  its  harmlessness  and  lower  cost 
compared  to  other  sensors  such  as  laser  range  scanners. 

Stereo  vision  is  an  attractive  technique  for  depth  perception,  which  is  suitable  in 
almost  all  application  domains  [1],  In  stereo  vision,  matching  points  are  first  extracted 
from  the  stereo  image  pair  by  observing  the  changes  of  the  ambient  light  reflection 
or  radiation  from  the  object  [2].  Depth  of  the  object  can  then  be  derived  directly  by 
matching  these  extracted  points  through  triangulation  method.  Depth  sensing  using 
stereo  vision  does  not  depend  on  the  projection  of  active  light  source.  Lighting  of  the 
working  environment  for  stereo  vision  does  not  need  to  be  well  controlled.  Hence, 
stereo  vision  is  useful  for  more  general  vision  applications  [ij.  However,  the  operation 
of  stereo  vision  relies  on  the  changes  of  the  light  reflection  or  radiation  from  the  object 
surfaces.  For  objects  made  of  metal  plates,  it  is  difficult  to  use  stereo  vision  to  detect 
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its  smooth  surface  since  the  smooth  surface  will  not  generate  matching  features  in 
the  image  pair.  This  inherent  problem  prevents  stereo  vision  from  being  widely  used 
in  industrial  automation  applications  where  the  objects  to  be  detected  are  smooth. 

Active  ranging  methods  by  the  projection  of  structured  light  provide  solutions  to 
the  inherent  problem  of  stereo  vision  in  detecting  smooth  surfaces.  In  essence,  ranging 
methods  using  structured  light  are  still  stereo  methods.  If  only  one  camera  is  used, 
depth  of  the  object  can  be  derived  by  matching  the  created  features  on  the  surface  of 
the  object  to  the  features  extracted  from  the  acquired  image  through  triangulation. 
A  typical  work  using  this  method  can  be  seen  in  [3].  Alternatively,  a  pair  of  cameras 
can  be  used.  In  such  cases,  depth  of  the  object  is  derived  by  using  exactly  the  same 
method  in  stereo  vision  except  that  the  matching  features  in  the  image  pair  are  created 
by  the  projection  of  structured  light.  By  using  structured  light,  it  is  easier  to  extract 
image  features  from  the  image  pair.  Usually,  binary  images  can  be  generated  when 
the  illumination  of  the  working  environment  is  well  controlled.  Most  importantly, 
the  correspondence  problem  in  stereo  vision  becomes  simpler  since  the  pattern  of  the 
matching  features  in  the  stereo  images  are  more  regular  following  the  pattern  of  the 
light  source  [3].  The  ranging  method  using  structured  light  is  named  “active  stereo 
vision”.  Conversely,  conventional  stereo  vision  without  the  usage  of  structured  light 
is  referred  to  as  “passive  stereo  vision”  in  this  paper. 

The  correspondence  problem  does  not  exist  in  active  stereo  vision  if  a  single  light 
beam  or  a  single  light  plane  is  used  as  the  light  source.  Only  the  brightest  spot  or 
curve  in  the  images  generated  by  the  light  beam  or  the  light  plane  respectively,  need  to 
be  detected  and  matched.  As  successive  scanning  will  be  required  to  detect  the  whole 
surface,  sensing  process  will  become  slow.  If  a  grid  of  light  is  used  as  the  light  source, 
there  will  be  more  data  points  in  one  single  detection,  and  the  matching  of  the  created 
features  in  the  image  pair  will  become  complicated.  The  matching  problem  can  be 
solved  by  imposing  unique  properties  on  the  stripes  of  the  light.  Approaches  using 
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colors  [4],  thickness  [5],  space  encoding  [6,  7]  have  been  suggested  and  implemented. 
These  approaches  require  special  devices  (color  or  thickness  detection)  and  require 
.miltiframes  (space  encoding),  and  are  thus  not  convenient  for  implementation.  It  is 
easier  to  generate  and  detect  a  uniform  grid  of  light.  However,  matching  the  features 
created  by  the  uniform  grid  of  light  is  a  difficult  task.  Such  a  task  has  been  studied 
by  Hu  and  Stockman.  They  used  constraint  propagation  to  solve  such  a  matching 
problem  in  active  stereo  vision,  assuming  that  the  spatial  frequency  of  the  surface  is 
less  than  the  stripe  frequency  [3]. 

The  algorithms  provided  by  Hu  and  Stockman  for  matching  grid  points  in  the 
image  plane  to  the  grid  points  in  the  projection  plane  are  complicated.  The 
complication  of  their  algorithms  are  mainly  caused  by  the  fact  that  the  surfaces  of  the 
objects  might  be  occluded  in  general  cases  and  it  is  difficult  to  distinguish  x  stripes 
from  y  stripes  in  the  image  plane.  Furthermore,  due  to  blooming  effects,  surface 
discontinuities,  and  the  difference  in  the  viewing  angle  of  the  cameras,  some  of  their 
topological  constraints  for  matching  the  grid  points  may  not  hold  [3].  In  consequence, 
it  is  difficult  to  identify  the  grid  point  in  the  projection  plane  corresponding  to  a 
particular  grid  point  in  the  image  plane.  Several  matchings  between  the  grid  points  in 
the  image  plane  and  the  projection  plane  can  be  resulted  by  applying  their  algorithms. 
In  order  to  check  which  matching  combination  is  the  correct  one,  the  deduced  surface 
solution  from  a  particular  matching  has  to  be  back  projected  to  the  projection  plane. 
Their  algorithms  are  therefore  time  consuming. 

In  tli is  study,  passive  stereo  vision  is  proposed  to  detect  the  boundary  of  the 
surface  of  the  deformed  plates.  Active  stereo  vision  with  the  projection  of  structured 
light  is  used  to  detect  the  details  of  the  surface.  The  two  cameras  and  the  projector 
are  proposed  to  be  mounted  on  movable  platforms  such  as  robot  manipulators. 
The  cameras  and  the  projector  can  thus  be  moved  to  appropriate  positions  and 
orientations  for  easy  detection  and  matching  the  features  created  by  the  projection 
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the  smooth  surface.  Such  a  dynamic  sensing  mechanism  is  also  necessary  for  the 
detection  of  3-D  objects  in  3-D  space  since  the  complete  description  of  3-D  objects 
can  only  be  obtained  by  sensing  the  objects  from  multiple  views. 

A  particular  problem  in  stereo  vision  has  to  be  considered  in  applying  passive  and 
active  stereo  vision  for  the  detection  of  smooth  surfaces.  When  the  surface  is  curved, 
the  actual  boundary  of  the  surface  may  be  hidden  from  the  cameras.  As  a  result,  the 
observed  boundary  from  passive  stereo  vision  is  not  accurate.  This  problem,  named 
the  “false  boundary”  problem  in  this  paper,  will  also  complicates  feature  matching 
in  active  stereo  vision. 

The  viewing  direction  and  position  of  the  stereo  vision  sensor  affects  the  occurrence 
of  the  false  boundary  problem  in  stereo  vision.  Since  the  extent  of  the  smooth  surf,  iv 
is  not  available,  the  pose  of  the  stereo  vision  sensor  can  hardly  be  devised  to  prevent 
the  false  boundary  problem.  The  proposed  dynamic  sensing  allows  the  sensor  and 
the  projector  be  moved  to  better  positions  to  achieve  accurate  sensing  based  on  the 
information  obtained  from  previous  sensing,  and  prevent  the  false  boundary  problem. 

In  order  to  test  the  effectiveness  of  the  proposed  approach  using  integration  of 
passive  and  active  stereo  vision  for  the  detection  of  smooth  surfaces,  an  experiment 
was  conducted.  A  curved  rectangular  plate  is  considered  as  the  target  for  shape 
detection. 

In  the  following  sections,  we  will  first  explain  the  so-called  “false-boundary” 
problem  in  stereo  vision  in  Section  II.  Then,  the  matching  criteria  for  active  stereo 
vision  in  the  case  without  the  occurrence  of  false  boundary  problem  is  described  in 
Section  III.  The  process  of  dynamic  sensing  which  integrates  passive  and  active  stereo 
vision  for  detecting  smooth  surfaces  is  presented  in  Section  IV.  The  experiment  is 
described  in  Section  V.  Conclusions  and  some  discussions  are  given  at  the  end  of  this 
paper. 
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Figure  1:  Two  examples  of  the  false-boundary  problem  in  stereo  vision. 

II.  False-Boundary  Problem  in  Stereo  Vision 


When  a  curved  surface  is  seen  by  the  two  cameras  of  the  stereo  vision  sensor,  the 
boundary  of  the  surface  detected  in  each  camera  might  not  correspond  to  the  actual 
boundary  of  the  surface.  As  a  result,  the  boundary  of  the  surface  recovered  from 
matching  these  two  “false  boundaries”  in  the  two  cameras  by  triangulation  will  be 
incorrect.  Shown  in  Fig.  1  are  two  examples  of  this  problem.  In  (a)  of  Fig.  1,  the 
right  side  of  the  surface  boundary  is  hidden  from  the  left  camera  and  the  left  side  of 
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Figure  2:  A  mismatch  of  the  created  features  on  the  surface.  The  “x”  symbols  stand 
for  the  created  features. 

the  surface  boundary  is  hidden  from  the  right  camera.  In  (b)  of  Fig.  1,  the  lower  side 
of  the  surface  boundary  is  hidden  from  the  two  cameras. 

The  false-boundary  problem  also  affects  the  correctness  of  feature  matching  in 
active  stereo  vision.  Fig.  2  shows  an  example  of  feature  mismatching  in  active 
stereo  vision.  In  Fig.  2,  four  features  are  created  by  the  structured  light  on  the 
surface.  However,  each  camera  can  only  see  three  of  the  four  created  features.  If  the 
three  features  in  the  image  pair  of  the  stereo  vision  are  matched  to  one  another,  the 
recovered  depth  corresponding  to  the  features  on  the  surface  will  be  incorrect. 

There  are  two  ways  to  eliminate  the  occurrence  of  the  false  boundary  problem  in 
stereo  vision.  One  is  to  orient  the  center  line  of  the  stereo  vision  sensor  in  order  that 
the  center  line  will  bisect  and  be  perpendicular  to  the  line  segment  connecting  the 
two  end  points  of  the  most  bending  curve  drawn  on  the  surface.  Such  arrangement 
of  the  center  line  will  reduce  the  likelihood  that  the  surface  boundary  is  hidden  from 
the  views  of  the  cameras.  The  other  is  to  move  the  stereo  vision  sensor  farther  away 
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(a)  (b) 

Figure  3:  (a)  The  false  boundary  problem  occurs,  (b)  The  center  line  of  the  stereo 
vision  sensor  is  bisecting  and  perpendicular  to  the  line  segment  connecting  the  two 
end  points  of  the  most  bending  curve  on  the  surface.  The  false  boundary  problem 
may  be  eliminated. 

from  the  surface.  In  this  way,  the  view  angle  of  the  cameras  formed  by  the  curved 
surface  will  be  decreased  and  may  eliminate  the  false  boundary  problem.  These  two 
approaches  are  illustrated  in  Fig.  3  and  4,  respectively.  However,  if  the  surface  is  so 
curved  that  the  projection  of  its  actual  boundary  in  the  image  plane  is  within  the 
projection  of  its  apparent  boundary,  as  shown  in  Fig.  5,  the  false  boundary  problem 
can  not  be  resolved  by  these  two  approaches.  In  such  a  case,  it  is  necessary  to  detect 
the  surface  from  multiple  views  of  the  stereo  vision  sensors.  The  latter  case  will  not 
be  emphasized  in  this  study.  The  approaches  developed  in  this  paper,  however,  can 
be  easily  extended  and  become  applicable  to  the  case.  We  only  need  to  apply  the 
approach  several  times  from  different  directions  of  the  stereo  vision  sensor. 
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Figure  4:  (a)  The  false  boundary  problem  occurs,  (b)  The  stereo  vision  sensor  is 
moved  farther  away  from  the  surface  to  prevent  the  false  boundary  problem. 

III.  Matching  Criteria  in  Active  Stereo  Vision 

In  this  section,  we  discuss  the  criteria  for  matching  the  features  in  the  image  pair 
in  the  case  where  the  false  boundary  problem  does  not  exist.  The  features  created 
by  the  structured  light  are  chosen  to  be  an  array  of  squares,  as  shown  in  Fig.  6.  The 
corners  of  the  squares  can  be  used  as  matching  points  for  deriving  the  data  points 
in  3-D  space.  Squares  are  chosen  because  they  provide  more  matching  points  than 
other  simple  features  such  as  circles  and  triangles.  The  detection  of  the  corners  of 
squares  is  more  reliable  than  that  of  more  complicated  features  such  as  pentagons 
and  hexagons.  In  the  case  that  the  detection  of  the  corners  is  not  accurate  when  the 
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Figure  5:  A  case  when  the  surface  has  to  be  detected  from  multiple  views. 

lighting  is  not  well  controlled,  the  geometric  centers  of  the  features  should  be  chosen 
as  the  matching  points.  In  such  cases,  simpler  features  are  preferred.  Considering 
the  convenience  for  solving  the  correspondence  problem,  we  chose  the  conventional 
imaging  geometry  for  the  use  of  stereo  vision.  Such  geometry  involves  a  pair  of 
cameras,  with  their  optical  axes  and  image  planes  mutually  parallel  and  separated  by 
a  horizontal  distance  denoted  as  the  stereo  baseline. 

Fig.  7  shows  an  illustration  of  the  case  well  defined  to  help  explain  the  matching 
criteria  in  active  stereo  vision.  A  surface  is  assumed  to  be  flat  and  parallel  to  the 
image  planes  of  the  cameras.  An  imaginary  plane,  named  “structured  light  plane” 
in  this  paper,  is  defined  as  the  intersection  of  the  structured  light  and  an  arbitrary 
plane  perpendicular  to  the  center  ray  of  the  structured  light.  The  center  ray  of  the 
structured  light  is  oriented  parallel  to  the  optical  axes  of  the  cameras,  while  the 
structured  light  plane  is  oriented  parallel  to  the  image  planes  of  the  cameras.  The 
created  matching  squares  on  the  structured  light  plane  will  then  lie  parallel  to  the 
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Figure  6:  The  features  to  be  created  by  the  structured  light  in  active  stereo  vision. 

i  and  y  axes  of  the  structured  light  plane.  The  orientation  of  the  matching  squares 
is  chosen  the  same  as  that  of  the  structured  light  plane.  For  the  convenience  of  the 
following  discussion,  we  denote  each  row  of  squares  positioned  in  the  structured  light 
plane  as  a  “horizontal  square  array”  in  the  structured  light  plane.  In  the  image 
plane,  the  squares  projected  from  a  horizontal  square  array  in  the  structured  light 
plane  are  referred  to  as  “horizontal  square  array”  in  the  image  plane. 

The  uniqueness  constraint  and  continuity  constraint  identified  by  Marr  and  Poggio 
[8]  can  be  utilized  for  matching  the  squares  in  such  a  well  defined  case.  According  to 
the  uniqueness  constraint,  there  will  be  only  one  square  in  the  images  of  the  cameras 
corresponding  to  one  square  on  the  flat  surface.  Adjacent  squares  in  the  image  will 
be  projected  from  adjacent  squares  on  the  flat  surface  if  continuity  constraint  is 
imposed.  With  the  use  of  epipolar  constraint,  the  squares  in  horizontal  square  arrays 
in  the  image  planes  can  be  matched  according  to  their  y  coordinates  in  the  image 
planes.  With  the  use  of  the  uniqueness  and  continuity  constraints,  each  square  in 
the  horizontal  square  array  in  the  image  pair  can  be  matched  according  to  their 
relative  positions  in  the  x  direction.  The  matching  criteria  described  above  is  not 
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Figure  7:  A  well  defined  case  for  explaining  the  matching  criteria  in  active  stereo 
vision.  Symbols  ”x"  stand  for  the  matching  squares. 

restricted  to  the  case  of  rectangular  surfaces  shown  in  Fig.  7.  In  cases  that  the 
surface  is  not  rectangular,  the  numbers  of  squares  in  the  horizontal  square  arrays  will 
not  be  the  same.  The  matching  criteria  described  above  are  still  applicable  to  these 
noM-rectangular  cases. 

In  reality,  a  deformed  plate  may  have  a  curved  surface.  As  a  result,  the  positions 
of  the  squares  in  the  image  pair  will  be  shifted  from  their  positions  projected  in  the 
well  defined  case.  However,  shown  in  the  image  pair,  the  relative  positions  in  the  x 
direction  of  the  squares  projected  from  a  horizontal  square  array  in  the  structured  light 
plane  are  preserved.  Similarly,  the  relative  positions  in  the  y  direction  of  the  horizontal 
square  arrays  are  also  preserved.  Since  the  spacing  between  adjacent  squares  in  the 
.r  direction  in  the  structured  liglil  can  be  designed  to  be  sufficient!}'  small,  we  can 
assure  that  two  adjacent  squares  in  the  x  direction  in  the  structured  light  plane  will 
be  projected  to  the  image  plane  as  two  adjacent  squares  in  the  x  direction  with  their 
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Figure  8:  (a)  A  required  projection  of  two  “horizontal  square  arrays”  in  the  structured 
light  plane  to  the  image  plane,  (b)  A  projection  that  should  be  avoided.  In  (a)  and 
(b),  squares  A  and  B  are  projected  from  the  same  horizontal  square  array,  and  squares 
C  and  D  are  projected  from  another  horizontal  square  array. 

y  coordinates  overlapping.  Furthermore,  the  spacing  between  two  adjacent  horizontal 
square  arrays  in  the  projection  plane  can  be  designed  sufficiently  large  in  order  that  a 
square  in  the  image  plane  projected  from  a  particular  “horizontal  square  array”  in  the 
structured  light  plane  will  be  adjacent  to  squares  projected  from  the  same  horizontal 
square  array  in  the  structured  light  plane.  These  tw'o  conditions  are  explained  in  Fig. 
8. 

Based  on  the  above  description  about  the  positional  arrangement  of  the  matching 
squares  in  the  image  pair  of  stereo  vision,  matching  of  the  squares  can  be  stated  as 
follows.  First,  the  squares  in  each  image  are  sorted  into  groups,  corresponding  to  the 
horizontal  square  arrays  in  the  structured  light  plane.  The  sorting  of  the  squares  can 
be  performed  by  applying  the  following  algorithm  to  the  image  pair: 
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1.  Start  from  an  arbitrary  square  extracted  from  the  image.  Denote  this  square 
as  the  current  square. 

2.  Search  the  square  in  the  image  that  is  left  adjacent  to  the  current  square.  If 
searched,  link  it  to  the  left  of  the  current  square  and  denote  it  as  the  left  square. 

3.  If  the  left  square  is  found,  search  the  square  that  is  left  adjacent  to  the  left 
square,  link  the  square  to  the  left  of  the  left  square,  and  denote  it  as  the  new 
left  square.  Repeat  this  step  until  there  is  no  square  that  is  left  adjacent  to  the 
left  square. 

4.  Search  the  square  in  the  image  that  is  right  adjacent  to  the  current  square.  If 
searched,  link  it  to  the  right  of  the  current  square  and  denote  it  as  the  right 
square. 

5.  If  the  right  square  is  found,  search  the  square  that  is  right  adjacent  to  the  right 
square,  link  the  square  to  the  right  of  the  right  square,  and  denote  it  as  the  new 
right  square.  Repeat  this  step  until  there  is  no  square  that  is  right  adjacent  to 
the  right  square. 

6.  Choose  an  arbitrary  square  from  the  squares  that  have  not  been  linked  to  other 
squares.  Denote  it  as  the  new  current  square. 

7.  Repeat  step  2  to  step  6  until  there  is  no  more  square  left  in  the  image. 

After  the  squares  are  sorted,  groups  in  the  image  pair  can  be  matched  according  to 
the  y  coordinate  of  their  leftmost  squares.  Each  square  of  a  matched  group  in  the 
image  pair  can  be  matched  according  to  their  relative  positions  in  the  x  direction. 

Based  on  the  above  description,  the  conditions  that  will  ensure  successful  feature 
matching  are  summarized  as  follows. 
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1.  First,  the  projector  should  be  oriented  in  such  a  way  that  the  center  ray  of  the 
structured  light  is  parallel  to  the  optical  axes  of  the  cameras  and  the  structured 
light  plane  is  parallel  to  the  image  planes  of  the  cameras  (Fig.  7). 

2.  Second,  according  to  the  matching  criteria,  it  is  required  that  the  regions  where 
the  squares  are  to  be  matched  be  defined.  The  regions  can  be  defined  by 
detecting  the  surface  boundary  in  the  image  pair. 

3.  Third,  the  two  conditions  shown  in  Fig.  8  regarding  the  relation  of  the  squares 
in  the  image  planes  should  be  satisfied.  To  ensure  these  two  conditions  the 
spacing  between  adjacent  squares  in  the  x  direction  in  the  structured  light 
plane  should  be  designed  sufficiently  small  and  the  spacing  between  adjacent 
horizontal  square  arrays  should  be  designed  sufficiently  large.  However,  it  is 
not  always  easy  to  estimate  the  required  spacing  between  two  adjacent  squares 
in  the  structured  light.  To  further  ensure  these  two  conditions,  the  structured 
light  plane  and  the  image  planes  should  be  oriented  perpendicular  to  the  normal 
vector  of  the  boundary  plane.  The  boundary  plane  is  defined  to  be  the  best -fit 
plane  of  the  detected  boundary.  That  is,  the  overall  distance  from  the  detected 
boundary  to  the  boundary  plane  is  minimum.  Mathematically,  the  boundary 
plane  should  minimize  the  following  integral: 

D[s)d s, 

where  D(s)  is  the  distance  from  a  point  s  on  the  boundary  to  the  plane.  The 
result  of  this  arrangement  is  that  the  average  normal  vector  of  the  surface  is 
more  likely  to  be  in  the  same  direction  as  the  optical  axes  of  the  cameras  and  the 
disparity  in  the  y  direction  of  adjacent  squares  in  the  image  planes  is  decreased. 
In  consequence,  the  likelihood  of  the  false  boundary  problem  as  shown  in  Fig. 

1  and  the  mismatch  problem  as  shown  in  Fig.  8(b)  is  reduced.  Mathematical 
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proof  of  this  advantage  is  out  of  the  scope  of  this  paper  and  will  not  be  discussed 
here. 

A  special  case  should  be  considered  in  identifying  the  matching  squares  in  the 
image  planes.  In  the  case  that  the  squares  are  projected  from  the  structured  light  on 
the  boundary  of  the  surface,  the  squares  in  the  image  pair  will  be  greatly  distorted. 
Their  shapes  could  become  triangular,  quadrilateral,  pentagonal,  etc.  and  their  size 
will  become  smaller.  A  criterion  is  needed  to  treat  these  “partial”  squares  as  either 
matching  features  or  as  noisy  features.  The  design  of  such  a  criterion  should  be  based 
on  how  robust  the  image  processing  is  used  to  extract  the  features. 

IV.  Sensing  Process 

Two  kinds  of  calibration  must  be  performed  to  ensure  that  the  stereo  vision 
sensors  and  the  projector  can  be  applied  to  detect  the  curved  surface  in  dynamic 
and  exploratory  sensing.  One  is  the  robotic  hand/eye  calibration  of  the  stereo  vision 
sensors  mounted  on  the  robot  arm.  The  hand/eye  calibration  is  to  obtain  the  relative 
position  and  orientation  between  the  robot  hand  and  the  cameras.  As  a  result,  the 
images  of  the  cameras  can  be  used  to  precisely  measure  the  position  of  a  target.  The 
other  is  the  structured  light  calibration,  designed  to  obtain  the  equations  that  describe 
the  structured  light  in  the  coordinate  system  of  the  robot  arm  holding  the  projector. 
Based  on  the  results  of  the  structured  light  calibration,  the  position  and  orientation 
of  each  structured  light  ray  generating  the  matching  squares  on  the  structured  light 
plane  can  be  derived  in  the  coordinate  system  of  the  robot  arm  holding  the  projector. 
The  structured  light  calibration  can  be  performed  by  using  the  same  method  for  the 
hand/eye  calibration  of  the  stereo  vision  sensors.  A  simple  calibration  method  is 
provided  in  another  paper  (9]  and  will  not  be  further  addressed  here. 
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The  structured  light  calibration  serves  two  purposes.  During  the  process  of  active 
stereo  vision  sensing,  the  structured  light  should  be  oriented  to  the  desired  positions 
and  orientations  as  described  in  Section  III  to  ensure  successful  feature  matching.  The 
other  purpose  of  the  structured  light  calibration  is  to  help  detect  the  false  boundary 
problem  in  stereo  vision,  which  will  be  explained  later  in  this  section.  The  process 
of  dynamic  sensing  for  detecting  the  shape  of  the  curved/smooth  surfaces  including 
three  basic  stages  is  described  in  the  following. 

In  the  first  stage,  passive  stereo  vision  is  applied  to  obtain  a  rough  boundary 
of  the  surface.  In  the  literature,  there  are  many  approaches  proposed  to  solve  the 
matching  problem  in  passive  stereo  vision.  Matching  algorithms  based  on  edges  [10], 
areas  [11],  segments  [12],  curves  [13],  structural  descriptions  of  image  primitives  [14], 
and  regions  [15]  are  often  mentioned  and  used  [16].  However,  passive  stereo  vision 
is  not  the  emphasis  of  this  research.  We  consider  a  specific  working  environment 
that  will  simplify  the  implementation  of  passive  stereo  vision,  instead  of  a  general 
environment  considered  in  these  approaches.  The  working  environment  is  assumed  to 
be  dark  with  light  illuminated  from  the  top  to  the  black  ground.  As  a  result,  it  will 
be  easy  to  extract  and  recognize  the  boundary  of  the  targeted  surface  in  each  image 
of  stereo  vision.  However,  the  detected  boundary  of  the  surface  may  not  be  accurate 
because  the  false-boundary  problem  might  occur  and  the  surface  can  be  out  of  the 
range  of  the  effective  focal  length  of  the  cameras.  Such  a  rough  boundary  detected 
in  this  stage  provides  guidance  for  the  movement  of  the  cameras  in  the  second  stage 
of  sensing. 

In  the  second  stage  of  the  sensing  process,  the  vision  sensor  will  be  moved  to  a 
closer  position  to  the  surface  so  that  the  distance  between  the  cameras  and  the  surface 
will  be  within  the  effective  focal  length  of  the  cameras.  In  addition,  the  image  planes 
are  oriented  as  perpendicular  to  the  boundary  plane  as  described  in  condition  three 
in  Section  III.  The  surface  boundary  is  detected  more  accurate  by  applying  passive 
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stereo  vision  again.  The  boundary  detected  by  passive  stereo  vision  in  this  stage  will 
define  the  area  where  active  stereo  vision  should  be  applied. 

In  the  third  stage  of  the  sensing  process,  without  moving  the  cameras,  the 
projector  is  moved  to  a  proper  position  and  orientation  in  order  that  it  can  project 
the  structured  light  on  the  surface  and  satisfy  the  conditions  stated  in  the  previous 
section.  The  images  of  the  surface  are  acquired  by  the  two  cameras.  The  created 
matching  squares  within  the  surface  boundary  in  the  images  can  be  extracted.  The 
squares  can  be  mismatched  due  to  the  false  boundary  problem  as  explained  in  Section 
II.  The  existence  of  the  false  boundary  problem  can  be  detected  by  comparing  the 
number  of  squares  extracted  in  the  image  pair  to  the  actual  number  of  squares  within 
the  surface  boundary.  If  the  number  of  the  squares  within  the  surface  boundary  is 
larger  than  that  in  either  of  the  two  images,  the  false  boundary  problem  exists.  The 
actual  number  of  squares  within  the  surface  boundary  can  be  known  by  checking 
how  many  structured  light  rays  are  projected  within  the  surface  boundary  since 
the  position  and  orientation  of  the  structured  light  rays  are  all  known.  If  the  false 
boundary  problem  is  detected,  the  cameras  can  be  moved  far  enough  away  from  the 
surface  to  view  the  real  boundary  as  shown  in  Fig.  4. 

After  the  squares  in  the  image  pair  are  matched,  either  the  corners  or  the  centers 
of  the  squares  in  the  image  pair  can  be  chosen  as  the  match  points  for  deriving  the 
data  points  of  the  surface.  The  data  points  of  the  surface  can  be  chosen  as  the  points 
in  space  that  minimize  the  sum  of  their  distance  from  each  of  the  back-projection 
lines  which  correspond  to  the  match  points  in  the  image  pair. 
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Figure  9:  The  setup  for  the  experiment  in  reconstructing  t he  shape  of  a  plate 

V.  Shape  Detection  of  a  Deformed  Plate 

In  order  to  illustrate  the  proposed  approach  to  smooth  surface  detect 
integration  of  passive  and  active  steieo  vision,  an  experiment  was  conducted  to  do 
the  shape  of  a  deformed  plate.  Tin'  procedure  and  the  results  of  the  experiment 
described  in  this  sect  ion. 

.1 .  Thf  S(tuj>  of  the  Eijxrmitnt 

Shown  in  Fig.  9  is  the  picture  of  the  setup  of  the  experiment.  The  tanret 
this  experiment  is  a  red angulai  plate  with  1  1  .(>'>  cm  in  length  and  T.d*>  cm  in  v.  a 
"I  he  plate  has  a  Fright  white  color  and  i>  hent  hy  llF  such  that  a  curved  sm  l.n 
formed.  A  dark  background  is  chosen.  Such  a  design  of  the  woiking  enviionu 
simplifies  the  image  processing  for  the  passive  atid  active  steieo  vision.  An  on  :: 
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Figure  10:  The  structure  of  a  stereo  vision  sensor  with  conventional  imaging  geometry. 

projector  is  used  to  project  structured  light  on  the  plate.  One  TM-540  camera  is  used 
for  the  stereo  vision  and  mounted  on  a  PUMA  560  robot  arm.  The  left  and  the  right 
images  in  the  conventional  imaging  geometry  of  stereo  vision  are  acquired  by  moving 
the  camera  to  two  different  positions.  In  the  following  description,  this  single-camera 
stereo  vision  sensor  will  be  treated  as  if  there  were  two  cameras  mounted  on  the  robot 
arm. 

B.  Exploring  Strategy  for  the  Experiment 

Exploring  strategies  is  essential  for  the  three  stages  of  vision  sensing  described 
in  Section  IV.  The  missions  of  the  exploring  strategy  are  to  determine  the  pose  of 
the  vision  sensor  and  the  projector,  and  to  develop  the  path  planning  for  the  robot 
arms.  The  performance  of  the  vision  sensor  depends  on  the  pose  of  the  vision  sensor 
and  the  projector,  which  will  be  the  emphasis  of  this  section.  Path  planning  for  the 

robot  arms,  irrelevant  to  the  performance  of  the  vision  sensing,  is  not  discussed  in 

\ 

this  study. 
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The  determination  of  the  pose  of  the  stereo  vision  sensor  is  affected  by  many 
factors.  According  to  [17],  the  requirement  that  will  affect  camera  placement  include 
the  resolution,  focus,  field  of  view,  visibility,  view  angle,  and  prohibited  regions  of 
the  cameras.  These  requirements  should  be  satisfied  for  the  proper  operation  of  the 
cameras.  In  addition,  some  parameters  of  the  stereo  vision  sensor  should  be  considered 
in  developing  the  exploring  strategy.  In  the  case  of  a  conventional  imaging  geometry, 
these  parameters  that  will  determine  the  pose  of  the  stereo  vision  sensor  include  the 
pose  of  the  center  line,  the  orientation  of  the  baseline,  and  the  distance  between  the 
object  and  the  center  point  of  the  stereo  vision  sensor.  Fig.  10  shows  these  parameters 
in  the  structure  of  the  stereo  vision  sensor.  In  developing  the  exploring  strategy  for 
the  three  stages  of  vision  sensing,  we  are  mainly  concerned  about  these  parameters. 

The  exploring  strategy  for  the  sensing  process  in  detecting  the  targeted  plate  is 
developed  in  the  following.  In  the  following  discussion,  we  assume  that  the  length 
of  the  baseline  is  fixed  and  the  focal  lengths  of  the  lenses  of  the  cameras  are  on-line 
adjustable. 

1.  Strategy  for  the  First  Stage  of  the  Sensing  Process 

The  exploring  strategy  in  the  first  stage  of  the  sensing  process  should  be  developed 
in  a  way  that  a  rough  boundary  of  the  plate  can  be  sensed.  The  field  of  view  of  the 
cameras  should  cover  the  possible  extent  of  the  plate  to  fulfill  the  goal  of  obtaining  a 
rough  boundary  of  the  plate  for  the  stereo  vision  sensor. 

As  the  deformed  shape  of  the  plate  is  not  available,  the  possible  extent  of  the 
plate  is  not  known.  However,  by  moving  the  cameras  far  enough  away  from  the  plate, 
all  the  area  of  interest  in  the  workspace  can  be  covered  by  the  field  of  view  of  the 
cameras.  The  pose  of  the  center  line  and  the  orientation  of  the  baseline  can  be  chosen 
arbitrarily  since  only  rough  boundary  of  the  plate  is  needed  in  this  stage.  Based  on 
the  rough  boundary,  the  boundary  plane  can  be  determined. 
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2.  Strategy  for  the  Second  Stage  of  the  Sensing  Process 

With  the  rough  boundary  of  the  curved  plate  available,  the  three  parameters  of 
the  stereo  vision  sensor  can  be  chosen  to  prevent  the  likelihood  of  the  false-boundary 
problem  in  stereo  vision  as  much  as  possible.  When  the  center  line  of  the  stereo 
vision  sensor  is  oriented  passing  through  the  geometric  center  of  the  actual  plate 
boundary  and  perpendicular  to  boundary  plane,  the  false-boundary  problem  has 
the  least  likelihood  to  occur.  Once  the  center  line  was  determined,  the  baseline 
should  be  oriented  perpendicular  to  the  direction  where  the  plate  is  bent  most.  This 
arrangement  of  the  baseline  increases  the  similarity  of  the  shapes  seen  by  the  two 
cameras,  and  decreases  the  possible  occurrence  of  the  false-boundary  problem.  The 
distance  between  the  plate  and  the  center  point  of  the  stereo  vision  sensor  should  be 
determined  long  enough  for  the  field  of  view  of  the  cameras  to  cover  the  extent  of  the 
plate.  By  the  above  arrangement,  a  more  precise  boundary  is  found  from  the  stereo 
images.  Based  on  this  new  boundan-,  a  new  boundary  plane  can  be  defined. 

3.  Strategy  for  the  Third  Stage  of  the  Sensing  Process 

The  mission  of  the  exploring  strategy  in  this  stage  is  to  determine  the  pose  of  the 
structured  light  projector.  The  projection  of  the  structured  light  has  been  described 
in  Section  III.  The  center  ray  of  the  structured  light  should  pass  through  the  center 
of  and  perpendicular  to  the  boundary  plane  as  defined  in  the  second  stage. 

C.  Process  of  the  Experiment 

The  process  of  the  experiment  follows  the  three  stages  of  the  sensing  process 
described  in  Section  IV’.  Fig.  11  shows  the  process  of  these  three  stages. 

In  the  first  stage,  the  two  cameras  were  moved  to  obtain  a  rough  location  of  the 
plate  boundary.  The  position  and  orientation  of  the  cameras  were  decided  by  the 
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Figure  11:  The  process  of  the  vision  sensing  in  detecting  the  deformed  plate. 

exploring  strategy  in  the  first  stage  of  sensing  process  stated  in  subsection  B. 

In  the  second  stage,  the  stereo  vision  sensor  was  moved  to  the  location  and 
orientation  as  discussed  in  the  previous  section.  A  more  precise  boundary  is  defined. 
By  using  passive  stereo  vision  in  the  first  and  second  stages,  the  false  boundary 
problem  can  be  largely  eliminated. 

In  the  third  stage  of  sensing,  the  projector  was  moved  to  a  position  and  orientat  ion 
as  described  in  the  previous  subsection.  The  matching  squares  created  by  the 
structured  light  in  the  two  images  were  detected  and  matched.  The  geometric  centers 
of  the  four  corners  of  the  matching  squares  in  the  images  were  considered  as  the 
matching  points  for  deriving  the  range  data  of  the  plate.  The  data  points  of  the  plate 
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corresponding  to  the  match  points  in  the  two  images  were  derived  by  the  method 
described  in  Section  IV. 

D.  Image  Processing  Techniques 

The  image  processing  for  the  experiment  needs  to  detect  the  corners  of  the 
matching  squares  in  the  image.  Based  on  the  design  of  the  working  environment 
in  this  experiment,  the  images  in  the  passive  and  active  stereo  vision  can  all  be 
thresholded  into  binary  images.  Four  tasks  are  designed  to  accomplish  the  image 
processing  for  the  detection  of  the  corners  of  the  squares  in  the  image:  thesholding 
the  image,  boundary  extraction  of  the  squares,  contour  following  of  the  squares, 
and  corner  detection  of  the  squares.  The  implementation  of  the  last  three  tasks  is 
modified  from  the  methods  introduced  by  Cabrelli  and  Molter  [IS].  These  four  tasks 
are  described  in  the  following. 

1.  Thesholding 

The  purpose  of  this  task  is  to  obtain  the  region  projected  in  the  image  from  the 
plate  in  passive  stereo  vision  as  well  as  the  regions  projected  in  the  image  from  the 
matching  squares  in  active  stereo  vision.  The  bi-level  moment  preserving  thresholding 
technique  introduced  by  Tsai  [19]  was  implemented  for  this  task.  This  thresholding 
technique  has  two  advantages.  One  is  that  the  threshold  value  can  be  generated 
automatically.  The  other  is  that  this  technique  can  recover  the  ideal  image  from  a 
blurred  version  [19].  After  thresholding,  the  regions  of  the  squares  are  black  and  their 
environment  is  white. 
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2.  Boundary  Extraction 

In  this  task,  the  pixels  on  the  boundary  of  the  squares  are  detected  by  applying 
the  following  algorithm  on  every  pixel  in  the  thresholded  image: 

1.  If  the  pixel  is  black  and  its  4NNs  (four  nearest  neighbors:  left,  right,  up,  and 
down  neighbors)  are  also  black,  the  pixel  is  marked  as  an  interior  point. 

2.  In  any  other  cases,  the  color  of  the  pixel  remains  unchanged. 

After  this  algorithm  is  applied,  the  resulting  images  contain  a  set  of  closed  contours 
that  are  extracted  from  the  regions  of  the  matching  squares.  In  order  to  eliminate 
noisy  contours  or  pixels  in  the  image,  each  black  pixel  in  the  image  was  examined  to 
see  if  it  is  connected  to  an  interior  point.  If  it  is  not,  it  is  considered  to  be  a  noisy 
pixel  and  is  painted  white. 

The  right  thresholded  image  of  the  matching  squares  is  shown  in  Fig.  12  (a).  Its 
boundary-extracted  image  is  shown  in  Fig.  12  (b). 

3.  Contour  Following 

The  purpose  of  this  task  is  to  distinguish  the  boundary  points  of  each  square 
from  all  the  boundary  points  which  were  extracted  in  task  two  and  to  sort  these 
distinguished  boundary  points  according  to  the  order  of  their  positions  on  the  square 
boundary.  The  implementation  of  this  task  is  similar  to  that  in  [18]  and  is  not 
described. 

4.  Corner  Detection 

To  detect  the  corners  of  each  square  in  the  image,  we  first  compute  the  inner 
product  of  the  right  and  the  left  unit  tangent  vector  at  each  pixel  on  the  boundary 
of  the  square.  The  right/left  unit  tangent  vector  can  be  approximated  by  the  unit 
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Figure  12:  (a)  Right  thresholded  image  of  the  matching  squares.  (b)  Right 

boundary-extracted  image  of  the  matching  squares. 

direction  vector  of  the  line  that  best  fit  the  pixel  and  the  following/proceeding  K 
pixels.  These  two  lines  that  best  fit  the  K  + 1  pixels  are  found  by  using  the  eigenvector 
line  fitting  method  [20].  The  value  of  K  is  arbitrary.  In  this  experiment,  K  is  chosen 
to  be  eight. 

After  the  inner  products  of  all  the  boundary  pixels  are  computed,  a  threshold 
value  of  the  inner  product  can  be  set.  The  boundary  pixels  with  their  inner  products 
larger  than  the  threshold  value  are  eliminated.  As  a  result,  smooth  portions  of  the 
boundary  of  the  squares  are  eliminated.  The  thresholding  will  provide  several  intervals 
at  which  the  inner  products  of  the  pixels  are  smaller  or  equal  to  the  threshold  value. 
Each  corner  of  the  square  is  detected  to  be  the  pixel  with  minimum  inner  product 
within  each  interval. 
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Figure  13:  The  sensed  data  points  of  the  curved  rectangular  plate. 


E.  Results 

The  surface  of  the  plate  interpolated  from  the  sixteen  sensed  data  points  by  using 
energy  minimization  method  is  shown  in  Fig.  14.  These  sixteen  points,  including 
the  four  corners  of  the  plate  and  center  points  of  the  twelve  matching  squares,  are 
shown  in  Fig.  13.  Energy  minimization  interpolation  method  is  described  in  another 
paper  [21].  In  Fig.  14,  the  domain  of  the  plate  in  the  x-y  plane  is  assumed  to  be  a 
rectangular  since  the  bending  of  the  plate  in  the  x  and  y  direction  is  very  small  to  be 
negligible. 

Because  the  exact  position  and  shape  of  the  plate  are  not  known,  there  is  no  way 
to  measure  the  accuracy  of  the  sensed  data  points  and  the  interpolated  surface.  A  test 
was  conducted  to  estimate  the  accuracy  of  the  passive  stereo  vision.  Passive  stereo 
vision  was  applied  to  detect  the  positions  of  the  corners  of  a  square  on  the  target 
block  that  was  used  in  hand/eye  calibration.  The  actual  coordinates  of  the  corners 
of  the  square  on  the  target  block  are  known.  By  comparing  the  detected  positions 
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Figure  14:  The  interpolated  surface  of  the  deformed  rectangular  plate. 

and  the  actual  positions  of  these  corners,  the  accuracy  of  the  passive  stereo  vision 
that  was  used  in  this  experiment  can  be  roughly  estimated.  The  error  in  the  depth 
( z  coordinate  in  the  coordinate  system  of  the  target  block)  is  within  2  mm  and  the 
error  in  the  x  and  y  coordinates  is  within  1  mm  at  a  distance  of  about  40  cm. 

VI.  Conclusions  and  Discussions 

Integration  of  passive  and  active  stereo  vision  is  introduced  in  this  paper  for  shape 
detection.  This  approach  is  designed  to  detect  the  shape  of  curved/smooth  surface 
of  metal  plates.  The  boundary  and  shape  are  first  roughly  identified  by  passive 
stereo  vision.  This  identification  helps  ease  the  correspondence  problem  in  active 
stereo  vision;  whereas  active  stereo  vision  detects  detailed  surface  of  the  object.  In 
particular,  this  passive  and  active  integration  approach  is  shown  effective  in  detecting 
and  solving  the  false-boundary  problem  embedded  in  the  shape  detection  of  the 


deformed  metal  plates.  An  exploring  strategy  is  developed  to  assist  such  a  sensing 
approach. 

The  three  stages  of  sensing  described  in  Section  IV  should  be  sufficient  for 
detecting  surfaces  that  are  not  seriously  deformed  such  as  those  targeted  in  Section 
V.  For  surfaces  that  are  bent  in  a  large  angle  as  shown  in  Fig.  5,  the  three  sensing 
stages  should  be  applied  in  different  directions  and  distances,  and  repeated  whenever 
needed.  The  development  of  the  exploring  strategy  for  the  movement  of  the  vision 
sensor  in  such  cases  will  be  much  more  complicated  than  that  in  the  case  described 
in  Section  V. 
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EXECUTIVE  SUMMARY 


Project  Statement  and  Objective 

This  research  investigates  digitizing  methodologies  for 
maintaining  ideal  sensor-to-surface  orientation  for  digitizing  an 
unknown  object.  A  digitizing  system  was  built  to  aid  in  this 
investigation.  In  this  system,  two  Laser  Displacement  Sensors  (LDS) 
are  position  during  the  digitizing  process  by  a  six  degree-of-freedom 
robot.  A  state-of-the-art  review  of  other  research  in  this  area  is 
presented  in  the  introduction.  Also,  an  extensive  list  of  companies 
marketing  Laser  Displacement  sensors  and  Digitizing  Systems  is  found 
in  the  appendix. 

Progress  to  date 

This  report  represents  the  work  accomplished  over  a  two  years 
period  culminating  in  a  Masters  Thesis  defense.  During  this  period,  a 
review  on  the  state-of-the-art  in  the  digitizing  technology  was 
conducted.  The  review  consisted  of  three  aspects.  The  first  aspect 
studies  the  current  technology  in  Laser  Displacement  Sensors.  The 
second  aspect  studies  the  methodologies  of  the  digitizing  process,  and 
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the  third  aspect  studies  the  surface  reconstruction  using  digitized 
discrete  data  points. 

From  studying  the  existing  technology,  maintaining  a  near 
normal  sensor-to-surface  orientation  was  found  to  be  a  key  issue  in 
obtaining  a  precise  reconstruction  of  an  unknown  surface.  Since  the 
surface  is  unknown,  however,  it  is  difficult  to  predict  such  an 
orientation.  A  new  methodology  is  proposed  to  achieve  a  precise 
measurement.  The  new  methodology  incorporates  a  six  degrees-of- 
freedom  robot  and  multiple  laser  displacement  sensors  (LDS).  Using 
multiple  LDS,  variations  in  surface  contour  can  be  locally  predicted. 
Using  this  result,  the  ideal  sensor-to-surfaced  orientation  can  be 
maintained.  This  report  details  the  system's  implementation  and 
describes  the  proposed  sensor-to-surface  orientation  techniques.  A 
detailed  error  analysis  is  presented  using  the  assumption  that  the 
unknown  surface  has  a  finite  maximum  curvature.  And  finally,  an 
experimental  verification  of  the  proposed  techniques  is  presented. 

Further  Work 

Future  work  will  examine  new  digitizing  approaches  which 
continue  to  scan  the  unknown  surface  as  planer  curve  but  removes  the 
restriction  on  the  LDS  orientation  vector  that  it  be  kept  within  a  plane. 
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CHAPTER  I 


INTRODUCTION 


In  recent  years,  digitizing  product  model  surfaces  has  become 
increasingly  important  to  the  manufacturing  industry.  The  digitizing 
technology  is  especially  important  when  computer-aided-design 
systems  fail  to  aid  in  the  design  process  of  product  surfaces.  For 
example,  in  the  automobile  industry,  draw  die  engineers  often  build  a 
ceramic  draw  die  model  in  order  to  design  the  draw  die  addendum 
surfaces.  The  addendum  surfaces  are  constructed  around  the  periphery 
of  the  model's  product  surfaces  (e.g.  an  automobile  body  part).  These 
addendum  surfaces  are  designed  to  control  the  draw  characteristics  of 
the  forming  sheet  metal.  If  these  surfaces  are  not  adequately  designed, 
the  sheet  metal  will  be  either  wrinkled  or  tom.  Due  to  their 
complexity,  the  addendum  surfaces  are  created  by  hand  on  the  ceramic 
models  [1].  The  engineer  uses  his  or  her  own  experience  to  determine 
the  "correct"  surfaces  to  use.  (Note,  the  surface  design  is  not  unique. 
This  can  be  observed  by  comparing  the  addendum  surfaces  designed  by 
different  engineers  for  the  same  product  surface  model. 
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The  designs  are  usually  significantly  different.)  Once  designed,  these 
surfaces  are  digitized  to  create  a  data  base  that  the  CAM  software  can 
use.  The  CAM  software  is  used  to  aid  in  manufacturing  a  prototype 
draw  die. 

In  other  cases,  even  the  product  surfaces  are  too  complicated  to 
be  designed  using  CAD  tools.  Instead,  the  product  surfaces  are  entirely 
designed  using  clay  or  ceramic  models.  To  manufacture  draw  dies  or 
injection  molds  for  these  kinds  of  products,  the  clay  or  ceramic  models 
must  first  be  digitized.  The  digitizing  data  can  then  be  used  to  generate 
programs  for  numerically  controlled  (NC)  machines  which  machine 
the  dies  or  molds. 

Another  application  of  the  digitizing  technology  is  in  "rapid  part 
acquisition"  or  so  called  "3-D  copying"  [2]  .  This  is  a  process  which 
reduces  the  time  and  cost  to  manufacturing  critical  parts  whose  CAD 
model  is  not  available.  Such  a  process  is  important  in  the  efficient  and 
effective  maintenance  of  valuable  equipment  utilized  in  both  military 
and  civilian  organizations. 

Although  the  digitizing  technology  is  increasingly  important  in 
today's  manufacturing  activities,  many  aspects  of  the  technology  still 
need  to  be  enhanced.  As  illustrated,  the  digitizing  process  needs  to  be 
fast,  accurate,  reliable,  nondestructive,  and  needs  to  produce  an 
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efficient  data  base  for  CAM  software  to  use.  In  the  past,  coordinate 
measuring  machines  (CMM)  using  touch  probes  have  been  used  for 
digitizing  sculptured  surfaces.  Digitizing  using  a  CMM  with  a  touch 
probe  is  accurate  and  reliable;  however,  it  is  also  slow.  This  has 
motivated  several  tool  manufacturers  to  replace  the  touch  probe  with  a 
Laser  Displacement  Sensor  (LDS).  In  addition,  3-axis  machining 
centers  are  now  being  used  to  position  the  LDS  while  digitizing.  See 
Appendix  A  for  a  list  of  tool  manufacturers  that  market  digitizing 
machines. 

The  LDS  can  collect  sculptured  surface  data  faster  than  a  touch 
probe.  In  addition,  the  LDS  does  not  require  mechanical  parts  to  come 
in  contact  with  the  sculptured  surface,  thus  protecting  the  surface  from 
possible  damage  or  movement.  However,  the  LDS's  accuracy  depends 
on  the  position  and  orientation  of  the  laser  with  respect  to  the 
sculptured  surface.  The  LDS  is  most  accurate  when  its  orientation  is 
nearly  normal  to  the  sculptured  surface  and  the  surface  is  within  the 
LDS’s  effective  stand-off  range. 

On  3-axis  machines,  the  LDS  is  held  in  a  fixed  orientation  while 
digitizing.  This  limits  the  types  of  sculptured  surfaces  that  can  be 
digitized.  Because,  the  LDS's  orientation  must  be  within  a  specified 
range  of  sensor-to-surface  perpendicularity,  only  sculptured  surfaces 
that  fall  within  that  range  can  be  digitized.  Work  has  been  done  to 
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increase  the  LDS's  surface-to-sensor  orientation  range  [3].  However,  by 
using  a  six  degree-of-freedom  robot  for  positioning  and  orienting  the 
LDS,  a  more  general  sculptured  surface  (one  with  high  tangential 
variations)  can  be  digitized.  Use  of  an  articulated  machine  is  suggested 
by  the  authors.  The  LDS's  position  and  orientation  can  then  be 
continuously  changed  during  the  digitizing  process.  Thus,  narrower 
variations  in  sensor-to-surface  orientation  can  be  maintained. 

To  study  the  issues  involved  with  maintaining  a  near 
perpendicular  orientation  between  the  sculptured  surface  and  LDS,  a 
non-contact  3-D  digitizing  system  with  six  degrees-of-freedom  was 
constructed.  The  system  uses  two  LDS’s  which  are  positioned  using  a 
robot  manipulator.  Two  different  approaches  to  maintaining  sensor- 
to-surface  orientation  while  digitizing  were  investigated,  namely  the 
Multiple  Beam  Approach  and  the  Extrapolation  Approach. 

The  structure  of  this  report  is  as  follows.  In  the  next  chapter,  we 
review  the  state-of-the-art  in  laser  digitizing  technology.  The  review 
will  include  three  main  topics:  LDS  technology,  digitizing 
methodologies,  and  surface  reconstruction.  The  first  topic,  LDS 
technology,  is  a  review  of  the  current  state-of-the-art  in  the  Laser 
Displacement  Sensor.  The  second  topic,  digitizing  methodologies,  is  a 
review  of  the  methodologies  for  digitizing  unknown  surfaces.  And, 
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the  third  topic,  surface  reconstruction,  is  a  review  of  methods  for 
representing  the  surface  based  on  the  digitized  data. 

The  rest  of  the  report  will  present  our  research  results.  Chapter  3 
outlines  the  mathematical  background  for  the  two  proposed  digitizing 
approaches.  Chapter  4  studies  the  theoretical  limitations  of  these  two 
approaches  given  that  the  unknown  surface  has  a  finite  maximum 
curvature.  Chapter  5  describes  the  hardware  and  software  used  to 
construct  the  digitizing  system  and  briefly  discusses  the  nonlinear 
positioning  accuracy  of  an  articulated  machine.  Chapter  6  presents  the 
experimental  results  of  the  proposed  digitizing  approaches.  Finally,  the 
report  condudeds  with  a  summary. 


CHAPTER  II 


DIGITIZING  TECHNOLOGY  REVIEW 

2.1  Introduction 

Digitizing  techniques  have  been  used  in  the  manufacturing 
industry  since  the  70s.  Automotive  and  aerospace  engineers  have 
used  digitizing  methods  to  produce  prototype  tooling  directly  from  clay 
models.  In  recent  years,  digitizing  surfaces  has  become  useful  in  more 
applications,  such  as  in  the  production  of  injection  molds,  die  molds, 
and  in  rapid  prototyping.  As  applications  for  digitizing  technology 
have  increase,  more  research  activities  have  been  conducted  to 
improve  the  precision,  efficiency,  and  flexibility  of  this  technology. 
The  research  is  especially  active  in  Japan  and  in  European  countries. 

In  the  earlier  years  and  still  today,  digitizing  systems  have  used 
touch  probes  for  their  measuring  sensor.  As  the  name  suggests,  touch 
probes  need  to  come  in  contact  with  the  object's  surface  in  order  for  the 
digitizing  system  to  take  a  single  measurement.  Due  to  the  slow 
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mechanical  nature  of  this  process,  measuring  a  large  number  of  data 
points  can  be  very  time  consuming  using  a  touch  probe.  In  recent 
years,  non-contact  measurement  sensors  have  been  introduced  which 
do  not  require  the  sensor  to  physically  come  in  contact  with  the  object's 
surface.  This  dramatically  decreases  the  time  needed  to  digitize  a 
surface.  A  number  of  optical  techniques  are  available  for  taking  non- 
contact  measurements.  Most  of  them  can  be  classified  into  three 
categories.  They  are  time-of-flight,  interferometry,  and  triangulation 
methods  [4]. 

The  time-of-flight  approach  is  effective  when  the  object  is  at  least 
several  meters  away  from  the  sensor  and  high  precision  is  not 
required.  The  interferometric  approach,  on  the  other  hand,  is  effective 
for  short  range  measurements  and  provides  resolution  in  the  range  of 
a  few  microns.  Both  time-of-flight  and  interferometric  approaches 
require  a  complicated  optical-mechanical  system  and  are  not  practical 
to  implement.  By  far,  the  most  popular  approach  is  the  triangulation 
approach.  The  triangulation  approach  is  simple  to  implement.  It  only 
requires  a  light  source  and  a  light  sensor  to  make  distance 
measurements  (the  principle  of  the  triangular  approach  will  be 
discussed  later  in  this  chapter).  Many  commercial  products  based  on 
the  triangulation  principle  are  available  on  the  market.  Among  them, 
a  device  called  a  laser  displacement  sensor  (LDS)  is  the  most  popular. 
In  such  a  device,  a  laser  beam  generated  by  a  laser  diode  serves  as  the 
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light  source  and  a  charged  coupled  device  (CCD)  sensor  serves  as  the 
light  sensor.  With  the  advent  of  the  laser  diode  technology,  a  LDS  can 
be  designed  in  compact  sizes.  These  small  housing  allow  the  LDS  to  be 
mounted  on  the  spindle  of  CNC  machines.  As  a  result,  CNC  machines 
can  be  easily  turned  into  a  digitizing  system.  In  addition,  the  LDS  can 
also  replace  the  touch  probe  on  a  coordinated  measurement  machine 
(CMM).  Increasing  the  digitizing  speed  has  been  the  main  advantage  to 
replacing  the  touch  probe  on  the  CMM. 

Overall,  the  digitizing  process  of  an  object  is  divided  into  two 
main  steps.  The  first  step  is  to  obtain  a  set  of  point  measurements  on 
the  object's  surface.  The  second  step  is  to  turn  this  set  of  data  points 
into  a  CAD  representation.  This  step  is  often  called  surface 
reconstruction.  Performance  of  the  first  step  is  measured  by  the  speed 
and  accuracy  in  obtaining  the  data  points  and  by  the  robustness  in  the 
digitizing  method,  i.e.,  the  range  of  surface  variations  that  can  be 
accurately  digitized.  Performance  of  the  second  step  is  measured  by  the 
efficiency  in  the  representation  of  the  reconstructed  surface  and  by  the 
efficiency  in  the  algorithm  which  converts  the  data  points  into  the 
reconstructed  surface. 

Research  directed  towards  improving  the  digitizing  process 
performance  is  actively  being  conducted  all  over  the  world. 
Improvement  is  being  achieved  in  three  different  areas.  The  first  area 
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improves  the  LDS  technology  by  designing  new  and  innovative 
structures.  The  goal  is  to  improve  the  resolution  of  the  LDS  and  to 
make  the  resolution  insensitive  to  surface  conditions.  The  second  area 
improves  the  digitizing  process  by  maintaining  sensor-to-surface 
orientation.  One  approach,  uses  image  processing  to  filter  the  data 
obtained  from  the  LDS.  This  method  is  called  the  center  of  gravity 
approach.  The  objective  is  to  increase  measurement  precision  by 
eliminating  the  noise  sensed  by  the  sensor.  Another  approach, 
artificially  varies  the  surface's  reflective  attributes  such  that  the  LDS 
can  digitize  even  transparent  surfaces.  This  method  is  called  the 
condensation-assisted  approach.  The  third  approach  discussed  in  this 
area  is  the  rotating-the-object  approach.  The  third  main  area  involves 
improving  algorithms  for  surface  reconstruction.  After  an 
introduction  to  the  triangulation  principle,  current  progress  made  in 
these  three  main  areas  will  be  reviewed. 

2.1  The  Principle  of  the  Triangulation  Approach 

In  the  triangulation  approach,  a  laser  beam  is  used  as  a  light 
source  to  generate  a  spot  on  the  object's  surface.  The  spot  is  detected  by 
a  light  sensing  device  (a  CCD  sensor  for  example)  through  a  focus 
lense.  As  the  distance  from  the  spot  to  the  light  source  varies,  the 
projection  of  the  spot  on  the  CCD  sensor  also  varies,  see  Fig.  2.1.  By 


identifying  the  projected  spot's  position  on  the  CCD  sensor,  the 
displacement  of  the  object  surface  can  be  calculated. 
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Figure  2.1  Principle  of  Triangulation  Approach 


The  precision  of  the  triangulation  approach  depends  on  a 
number  of  factors,  such  as,  the  resolution  of  the  CCD  sensor,  the  area  of 
the  spot,  and  the  reflectance  property  of  the  surface,  etc.  Considerable 
research  has  been  conducted  by  both  industry  and  academia.  These  will 
be  reviewed  in  the  rest  of  this  chapter. 

2.2  Research  on  the  LDS 

2.2.1  Improving  the  Existing  LDS  Performance 

The  performance  of  the  LDS  can  b^  improved  in  many  ways. 
Fundamentally,  the  diameter  of  the  laser  beam  should  be  as  small  as 
possible  so  that  the  light  source  will  ideally  activate  only  one  pixel  on 
the  sensing  device.  In  addition,  the  resolution  of  the  CCD  sensing 
device  should  be  as  small  as  possible.  This  requires  increasing  the  pixel 
density  on  the  CCD  sensing  device.  In  recent  years,  improvements  in 
these  two  areas  have  resulted  in  many  new  LDS  products  with 
outstanding  performance.  A  selected  list  of  LDS  products  (available  in 
the  North  American  market)  and  a  limited  summary  of  their 
specifications  are  shown  in  Table  2.1. 

As  shown  in  Table  2.1,  the  LDS  (CyberOptics  PRS-15)  with  the 
highest  resolution  (0.381  mm)  also  has  the  smallest  measuring  range 
(±0.15  mm).  A  small  light  spot  is  necessary  to  attain  the  higher 
precision.  However,  since  a  small  light  spot  generates  a  weak  light 


intensity,  the  CCD  sensor  must  be  close  to  the  spot  in  order  to  sense  it. 
As  a  result,  higher  precision  is  achieved  at  the  cost  of  a  smaller 
measuring  range.  This  general  observation  is  illustrated  by  comparing 
two  LDS  marketed  by  Keyence  (models  LB-70  and  LB-72).  Model  LB-70 
has  a  spot  diameter  of  1.0  x  2.0  mm,  a  resolution  of  40  mm,  and  a 
measuring  range  of  100  ±40  mm.  Whereas,  Model  LB-72  has  a  spot 
diameter  of  1.0  mm,  a  resolution  of  15  mm,  and  a  measuring  range  of 
40  ±  10  mm  [5]. 

Another  factor  that  affects  the  LDS's  precision  is  the  angle  of 
incidence  (or  angle  of  inclination)  of  the  laser  beam.  Clearly,  when  the 
light  beam  is  normal  to  the  object  surface,  the  light  spot  has  the 
smallest  area,  and  the  LDS  provides  the  highest  resolution.  When  the 
light  beam  angle  is  changed  from  its  normal  orientation,  the  spot  area 
increases.  As  illustrated  earlier,  this  enlarged  spot  area  decreases  the 
resolution  of  the  LDS.  For  example,  when  the  incident  angle  is  off  30 
degrees  from  a  normal  orientation,  the  Keyence  LDS  generates  an  error 
as  big  as  2%  of  the  measured  distance  [5].  This  error  is  too  large  for 
many  applications. 

Even  worse,  if  the  incident  angle  is  larger  than  45*  the  LDS  can 
fail  [3].  The  triangulation  approach  assumes  that  the  diffused  light 
reflected  from  the  object  surface  will  reach  the  CCD.  When  the 
orientation  of  the  laser  beam  is  not  normal  to  the  object  surface,  the 


Table  2.1  Select  Listing  of  Laser  Displacement  Sensors  available  in  the 
North  American  market. 


Company 


Part  No. 

Measuring 
Range  (mm) 

Resolution 

(mm) 

Response 
Time  (ms) 

Sensor  Size 
(sq-in) 

Aromat 

MQLAS4LA 

120V14 

40  ±10 

2.5 

25 

3.1 

MQLAS3LA 

C120VS10 

150  ±50 

600 

2 

3.1 

Chesapeake 

Laser  Systems 

LTG-2101 

254  ±2.5 

2 

0.6 

5.4 

LTG-2125 

635  ±38 

13 

0.6 

217 

CyberOptics 

PRS-15 

6.60  ±0.15 

0.381 

N/A 

7.4 

PRS-1600 

60.96  ±16 

40 

N/A 

4.6 

Keyence 

LB-72 

40  ±10 

15 

2 

2.0 

LB-70 

100  ±40 

40 

20 

2.0 

Micro  Switch 

\  HSV  100 

39  ±6 

6 

10 

47.6  f 

Renishaw 

OP5M 

50  ±10 

2.5 

15 

12.3 

Selcom 

2301 

95±4 

N/A 

N/A 

N/A 

2005 

1200  ±512 

N/A 

N/A 

N/A 

Medar 


MDC-250 


200  ±50 


116 


109.5 


amount  of  diffused  light  that  reaches  the  CCD  may  be  insufficient.  As  a 
result,  the  sensor  is  rendered  disabled.  However,  maintaining  a 
normal  orientation  to  an  unknown  surface  is  not  easy.  To  resolve  this 
problem,  researchers  have  looked  at  ways  to  make  the  LDS  less 
sensitive  to  surface  inclination.  Their  efforts  will  be  reviewed  in  the 
next  subsections. 

2.2.2  Reducing  LDS  Sensitivity  to  Surface  Inclination 

A  notable  research  effort  in  reducing  the  LDS  sensitivity  to 
surface  inclination  was  made  by  Saito  and  Miyoshi  of  Hokkaido 
University  in  Japan  [3].  They  proposed  a  multiple  lense  structure  to 
overcome  tne  sensitivity  problem.  The  structure  consists  of  a  He-Ne 
laser,  four  optical  lenses  and  three  CCD  sensors  as  shown  in  Fig.  2.2. 

One  lens  is  larger  in  diameter  than  the  other  three  and  is  called 
the  object  lens,  Li.  The  other  three  lenses  called  converging  lenses  are 
positioned  between  the  object  lens  and  a  CCD  sensor.  The  three 
converging  lenses  and  three  CCD  sensors  are  spaced  120*  apart  around 
the  optical  axis  of  the  object  lens.  The  CCD  sensor  (7  mm  by  2560  pixels) 
is  placed  at  the  focus  length  of  the  converging  lense,  f2-  The  laser  beam  is 
directed  through  the  center  of  the  object  lens  and  onto  the  target  surface. 
Some  of  the  diffused  light  reflected  off  of  the  target  surface  passes  back 
through  the  object  lens.  This  light  continues 
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Figure  2.2  Structure  of  Multiple  Lens  Approach 
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through  the  converging  lenses  and  is  finally  detected  at  the  CCD 
sensors.  From  the  position  of  the  sensed  light  on  the  CCD  sensors,  the 
distance  p  can  be  determined.  The  distance  p  is  the  distance  from  the 
object  lens  focus  length,  fi,  to  the  target  surface.  The  location  of  the 
spot  on  the  CCD  sensor,  d,  is  determined  by  the  following  equation: 


(2.1) 


Since,  {a,  fi,  {2)  are  constants  and  {d}  is  determined  by  the  CCD  sensor, 
the  unknown  displacement  value,  p,  can  be  calculated  from  Eq.  (2.1). 

Using  multiple  CCD  sensors,  this  LDS  structure  can  measure 
larger  contour  variations  in  the  target's  surface.  Due  to  the  position  of 
the  three  converging  lenses  and  CCD  sensors,  each  CCD  sensor 
measures  the  same  displacement,  p.  As  the  target  surface  inclination 
renders  one  or  two  of  the  CCD  sensors  disabled,  the  remaining  CCD 
sensors  can  continue  to  measure  the  displacement,  p.  In  this  way,  the 
LDS  becomes  less  sensitive  to  surface  inclination.  This  LDS  sensor 
design  is  reported  able  to  measure  target  surface  inclinations  up  to  70* 
[3].  Although,  this  new  LDS  increased  the  surface  inclination  range 
that  could  be  measured  without  a  complete  sensor  failure,  the  accuracy 
of  the  measurement  is  still  proportional  to  the  inclination.  This 
problem  was  not  addressed  by  the  authors. 


2.3  Improving  the  Digitizing  Process 


Several  companies  have  implemented  the  LDS  on  3  axis  CNC 
milling  machines,  see  appendix  A.  The  LDS  is  mounted  on  the 
machine's  spindle  where  a  cutting  tool  would  normally  be  mounted 
[6].  Because  of  the  machine's  limited  degree-of-freedom,  the  LDS  is 
maintained  in  a  fix  spindle-to-sensor  orientation  during  the  digitizing 
process.  Since  the  measurement  accuracy  is  a  function  of  the 
inclination  angle,  a  limited  range  of  surfaces  can  be  digitized  using  this 
method.  Recently,  two  different  efforts  have  been  made  to  improve 
the  precision  and  robustness  of  the  digitizing  process.  The  first  effort  is 
called  center  of  gravity  approach  which  processes  a  video  imagine  of 
the  CCD  sensor.  The  second  effort  artificially  modifies  the  object 
surface  conditions.  These  two  approaches  will  now  be  discussed  in 
more  detail. 

2.3.1  The  Center  of  Gravity  Approach 

More  advanced  technology  is  needed  to  implement  a  reduced 
laser  spot  area  method,  as  discussed  above,  for  improving  the  LDS's 
precision.  Due  to  this  technology  limitation,  an  "effective"  spot  area 
reduction  approach  has  been  developed,  called  the  center  of  gravity 
approach.  In  the  center  of  gravity  approach,  the  center  of  a  relatively 


large  spot  is  identified.  This  center  point  is  used  to  calculate  the 
displacement.  Using  this  approach,  the  LDS’s  resolution  is 
dramatically  improved. 

The  spot's  center  of  gravity  is  identified  using  a  moment 
computation  technique  developed  in  computer  vision  [7].  Accuracy  of 
the  calculated  center  of  gravity  is  within  a  single  pixel  on  the  CCD 
sensor.  As  a  result,  the  precision  can  be  improved  by  a  factor  of  ten.  As 
an  example,  using  this  approach  CyberOptics  Corp's  LDS  has  a 
resolution  ten  times  better  than  that  of  a  similar  LDS  made  by  Keyence 
[8]  which  uses  a  "traditional"  approach.  Because  an  image  processing 
unit  is  needed,  the  hardware  used  to  implement  the  center  of  gravity 
approach  is  more  expensive. 

In  [9],  Soucy  et  al.  further  study  the  case  when  the  spot  crosses 
over  a  reflectance  discontinuity.  When  the  laser  spot  crosses  over  a 
reflectance  discontinuity,  only  part  of  the  spot  is  likely  to  be  seen  by  the 
CCD  sensor.  Consequently,  there  will  be  an  error  between  the  center  of 
gravity  point  obtained  from  the  sensed  spot  and  the  actual  center  of 
gravity  point  on  the  surface.  In  order  to  better  understand  the 
performance  of  LDS  in  this  case,  a  model  is  presented  to  describe  the 
behavior  of  the  center  of  gravity  position.  First,  the  intensity  of  the 
laser  spot  is  assumed  to  be  a  2-D  Gaussian  distribution  and  the 
reflectance  discontinuity  is  an  ideal  step  function.  In  this  model,  the 


interpolation  error  is  proportional  to  the  integral  of  the  image  spot's 
variance  over  the  discontinuity.  This  study  reveals  that  the  center  of 
gravity  approach  has  significant  errors  when  the  laser  spot  crosses  over 
areas  with  reflectance  discontinuity.  Although,  an  error  analysis 
model  is  given  by  [9],  no  approach  is  proposed  to  cope  with  the 
discontinuity. 

Another  error  in  center  of  gravity  position  is  introduced  when 
the  incidence  angle  varies.  This  alters  the  relative  position  of  the 
center  of  gravity  within  the  spot.  Thus,  measurement  precision  is 
affected  when  the  incident  angle  is  not  zero. 

2.3.2  Condensation-Assisted  Approach 

As  just  discussed,  variations  in  the  light  reflectance  property  of 
the  object  surface  can  introduce  errors  in  the  LDS  measurements. 
When  the  object  surface  is  reflective  or  transparent  instead  of  diffuse, 
the  orientation  of  the  LDS  to  the  surface  is  more  critical,  because  the 
the  light  can  be  directed  away  from  the  CCD  sensor.  To  prevent  this 
problem.  Little  and  Vavreck  developed  a  condensation-assisted 
approach  to  artificially  modify  the  reflectance  property  of  the  surface 
[10].  In  the  condensation-assisted  approach,  a  thin  layer  of  water 
droplets  is  deposited  on  the  object  surface  by  directing  warm  moist  air 
at  the  laser  spot.  Here,  the  surface  takes  on  a  semi-diffuse  reflectance 
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property,  allowing  normal  operation  of  the  LDS.  As  a  result,  a  diffuse- 
type  LDS  can  be  used  to  measure  shiny  or  transparent  surfaces.  The 
disadvantage  of  this  approach  is  the  necessity  to  implement  a 
complicated  mechanical  system  for  water  deposit. 

2.3.3  Rotating  the  Object  Approach 

When  the  object  has  a  round  or  cylindrical  shape,  maintaining  a 
reasonable  inclination  angle  is  difficult.  An  alternative  approach  to 
digitizing  small  objects  with  these  shapes  has  been  investigated.  In  this 
approach,  the  LDS  is  kept  stationary  and  the  object  rotates. 

Saint-Marc  et.  al.  recently  developed  a  PC-based  range  finding 
system  composed  of  an  independent  laser  system  which  projects  a 
sheet  of  light  on  the  object  surface  [11].  The  object  is  mounted  on  a 
rotary  table  driven  by  a  personal  computer.  As  the  table  rotates,  the 
entire  area  of  the  object  can  be  digitized  while  maintaining  a  reasonable 
incident  light  angle. 

In  Japan,  the  same  rotating-object  approach  is  used  to  digitize  the 
profile  of  a  last  [12].  The  profile  data  is  integrated  into  a  CAD  system 
and  used  for  designing  shoes.  During  the  digitizing  process,  the  last  is 
rotated  while  the  LDS  is  moved  along  the  rotating  axis.  As  a  result,  the 
entire  last  can  be  digitized  with  high  precision. 
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2.4  Surface  Reconstruction  Techniques 

After  digitizing  a  surface,  the  set  of  discrete  data  points  needs  to 
be  converted  into  a  mathematical  representation.  The  mathematical 
representation  is  necessary  to  store  and  modify  the  surface  information 
in  a  convenient  and  efficient  manner.  In  general,  the  surface  can  be 
expressed  in  an  implicit  or  explicit  form  [13].  In  the  explicit  form,  the 
surface  height  is  expressed  as  a  function  of  the  surface  plane 
coordinates  (x,y): 

z  =  f(x,  y).  (2.2) 

In  the  implicit  form,  the  surface  is  expressed  as  the  function 

f(x,  y,  z)  =  0  (2.3) 

where  (x,  y,  z)  are  the  Cartesian  coordinates  of  the  surface  point.  The 
above  approaches  are  widely  used  in  computer  vision  for  recognizing 
objects. 

The  CAD  systems,  on  the  other  hand,  uses  a  parameterized  approach. 
In  general,  the  parameterized  form  expresses  the  surface  using  the 
following  equation: 
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X(u,  v)  =  (x(u,  v),  y(u,  v),  z(u,  v)),  (2.4) 

where  (u,  v)  are  parameters  within  the  range  [0,  1].  There  are  many 
parameterized  representations  in  CAD  systems  induding  Coon's  Patch, 
B-Spline,  and  Beizer-Surface  [14].  Since  they  are  all  well  defined 
approaches,  the  details  of  these  representations  will  not  be  addressed 
here. 


Difficulties  in  creating  a  mathematical  representation  for  the 
surface  fall  into  one  of  two  categories.  Either,  there  is  insuffident  data, 
or  too  much  redundant  data  in  the  set  of  digitized  data  points. 
Approaches  for  handling  these  two  cases  are  presented  below. 

2.4.1  Regularization  Approach 

The  regularization  approach  finds  a  surface  representation  from 
a  sparse  data  set.  Because  the  data  set  is  sparse,  a  surface  that  satisfies 
the  set's  constraint  is  not  unique.  This  approach  utilizes  a  stabilizing 
functional  S(s)  and  a  penalty  functional  P(s).  Here,  s  defines  the  point 
coordinates  on  the  surface.  The  penalty  functional  defines  a  cost 
associated  with  an  inexact  match  between  the  reconstructed  surface  and 
the  discrete  data  points.  The  stabilizing  function,  on  the  other  hand. 
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restricts  the  possible  surface  space.  Thus,  the  surface  reconstruction 
problem  requires  minimizing  the  following  energy  function 

E(s)  =  S(s)  +  P(s).  (2.5) 

Grimson  is  probably  the  first  one  to  give  a  rigorous 
mathematical  derivation  of  the  stabilizing  functional.  According  to  his 
"surface  consistency  theorem"  [15],  the  reconstructed  surface  should  be 
the  smoothest  surface  that  passes  through  the  data  points.  The 
stabilizing  functional  he  chose  is 

S(s)=JJ  [(sxx)2+2(sxy)2+(syy)2]  dx  dy.  (2.6) 

This  functional  is  the  strain  energy  of  a  thin  plate,  usually  called  "thin 
plate  spline".  The  penalty  functional  Grimson  chose  for  surface 
approximation  is 

P(s)=  b  I  [s(xi,  y j)  -  dj]2  (2.7) 

where  i  indicates  the  digitized  points.  By  minimizing  the  energy  as 
defined  above,  Grimson  was  able  to  find  a  smooth  surface  from  a 
sparse  set  of  data  [15]. 
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More  recently,  Muraki  et  al.  extended  the  regularization 
approach  by  introducing  more  constraints  [16],  which  included 
smoothness,  fitness,  and  contour  line  constraints.  These  constraints 
are  used  to  ensure  that  the  reconstructed  surface  smoothly 
approximates  the  known  elevation  values.  They  also  ensure  that 
surface  has  the  same  height  value  for  all  points  o.  contour  line. 
Thus,  the  reconstructed  surface  is  a  better  representation  of  the  original 
surface.  The  energy  functional  is  minimized  by  solving  a  large  linear 
system  of  simultaneous  equations.  The  authors  have  successfully 
reconstructed  a  detailed  3D  surface  by  applying  this  method  on  a  sparse 
data  set  from  a  known  surface. 

The  preceding  two  methods  convert  a  sparse  data  set  into  a 
dense  data  set.  However,  the  dense  data  set  is  still  not  in  a 
parameterized  representation.  Recently,  Chen  and  Zheng  [17] 
developed  a  new  approach  based  on  the  energy  minimization 
principle.  First,  they  used  a  Coon's  composite  patch  approach  to 
represent  the  unknown  surface.  Then,  using  the  energy  minimization 
approach,  the  coefficients  of  the  Coon's  patch  representation  are 
calculated.  As  a  result,  the  reconstructed  surface  is  represented  in  a 
format  compatible  with  CAD  systems. 

There  are  many  variations  to  the  regularization  approach; 
however,  all  are  based  on  the  energy  minimization  principle. 
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Although,  they  are  not  elaborated  here,  other  regularization 
approaches  can  be  found  in  [13]. 

2.4.2  The  Parametric  Grid  Construction  Approach 

In  comparison,  few  papers  are  published  on  the  subject  of  surface 
reconstruction  from  a  dense  set  of  data.  A  notable  work  in  the  area  is 
by  Vemuri  [18,  19].  He  developed  a  parametric  grid  approach  for 
finding  a  surface  representation  from  a  dense  set  of  digitized  data.  In 
his  approach,  a  parametric  or  computation  grid  {(si,  tp],  i  =  1, ...,  m,  j  =  1, 
...,  n,  is  first  constructed.  The  grid  is  constructed  in  such  a  way  that  for 
each  i  and  j,  there  exists  (si,tj)  that  satisfy  the  following  equation: 

(f](si,  tp,  f2(sj,  tp,  P(Si,  tp)  =  (xjj,  yjj,  Zjp  =  pij  (2.8) 

where  (xjj,  yjj,  Zjp  represents  the  dense  data  set  and  the  functions  fk(sj, 
tp,  k=l,2,3,  are  to  be  determined. 

The  parametric  grid  construction  yields  a  boundary  conforming 
curvilinear  coordinate  system.  Here,  some  coordinate  lines  are 
coincident  with  each  boundary  segment  and  the  other  curvilinear 
coordinates  vary  along  the  boundary  segment  monotonically.  With 
the  values  of  the  curvilinear  coordinates  thus  specified  on  the 


boundary,  the  values  in  the  Cartesian  coordinate  system  were 
generated,  one-to-one,  from  these  boundary  values. 
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The  increments  of  all  the  curvilinear  coordinates  were  defined 
to  be  unity.  Then,  these  coordinates  were  normalized  to  the  interval  [1, 
N],  where  N  =  n,  m.  More  precisely,  the  grid  points  s  and  t  were 
determined  in  the  following  way: 


1  n  (M  Pi+l,j-Rj  I  I) 

Si+i-si  +  n  HdMpk^j-pkjM) 


(2.9) 


and 


1  n  (II  pi,j+1-Pi/j  !  I) 

Vi-t|  +  n  .»(2iipi^1.pk)ii) 


(2.10) 


With  the  parameters  s  and  t  defined  above,  the  mapping  between  the 
Cartesian  coordinates  and  the  curvilinear  system  is  unique. 

Once  the  parametric  grid  is  constructed,  the  same  energy- 
minimization  principle  is  used  to  find  a  smooth  surface  representation 
which  minimizes  the  error  between  the  constructed  surface  and  the 
digitized  points.  If  the  digitized  data  measurements  are  accurate,  they 


can  be  used  directly  to  fit  CAD  surface  formats,  such  as  Coon's  patch,  B- 
spline,  or  Beizer  [18, 19]. 

2.5  Summary 

In  this  chapter,  we  have  reviewed  the  state-of-art  in  digitizing 
technology.  In  addition  to  the  works  mentioned  above,  research  efforts 
have  also  been  made  in  references  [20-44].  As  discussed,  the  current 
research  direction  can  be  grouped  in  the  three  aspects,  i.e.,  LDS 
technology,  digitizing  methodologies,  and  surface  reconstruction 
algorithms.  Overall,  the  progress  in  the  LDS  technology  is  more 
advanced  than  the  digitizing  methodologies.  That  is,  current  digitizing 
methodologies  do  not  adequately  utilize  the  high  precision  available  in 
LDS.  For  example,  Aromat  Corporation  of  Japan  has  released 
preliminary  specifications  claiming  their  LA300  series  LDS  (to  be  on  the 
market  in  1992)  has  a  resolution  of  0.2  mm  (8xl0*6  inch).  The  best 
mechanical  positioning  accuracy  is  far  less  sensitive  than  this.  Even 
when  the  positioning  resolution  is  high,  if  good  orientation  between 
the  LDS  and  the  surface  is  not  maintained,  the  LDS's  resolution  will  be 
degraded  or  rendered  disabled. 

To  improve  the  digitizing  process  performance,  we  propose  a 
new  approach.  The  approach  uses  a  six  degree-of-freedom  robot  to 
automatically  orient  the  LDS’s  beam  so  that  the  optimum  resolution  of 
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the  sensor  is  maintained.  Our  efforts  in  this  direction  are  reported  in 
the  remaining  chapters. 


CHAPTER  III 


MATHEMATICAL  DEVELOPMENT 

The  two  proposed  extrapolation  approaches  are  developed 
mathematically  in  this  chapter.  The  Multiple  Beam  Approach  is 
presented  first  followed  by  the  Polynomial  Extrapolation  Approach. 
The  purpose  of  these  approaches  is  to  estimate  (or  extrapolate)  the 
position  and  orientation  of  the  surface  at  the  next  point  to  be  digitized. 
This  information  is  used  to  position  and  orient  the  LDS  so  that  it 
maintains  a  near  perpendicular  orientation  to  the  surface  and  a 
constant  stand-off  distance  from  the  surface. 

3.1  Multiple  Beam  Approach 

The  multiple  beam  approach  consists  of  positioning  and 
orienting  a  set  of  LDS  which  simultaneously  collect  local  information 
about  the  surface.  The  gathered  information  is  then  used  to  extrapolate 
the  next  surface  point  to  be  digitized.  The  amount  and  type  of 
information  gathered  depends  on  the  number  of  LDS  in  the  set.  Using 
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two  LDS,  a  surface  point  along  with  a  surface  tangent  vector  through 
that  point  can  be  determined.  Using  three  LDS,  the  local  surface 
curvature  can  also  be  determined.  The  concept  of  the  Multiple  Beam 
Approach  using  two  LDS  can  be  explained  through  use  of  the 
illustration  in  Fig.  3.1. 


In  this  case,  two  LDS  are  mounted  as  a  set  with  their  laser  beams 
parallel  to  each  other  and  separated  by  a  short  distance,  b  (the  selection 
of  the  length  of  b  will  be  discussed).  The  global  position  and 
orientation  of  the  LDS  is  known.  This  is  the  case  since  LDS  is  assumed 
to  be  positioned  using  an  accurate  measuring  device.  The 


displacements  from  the  LDS  reference  position  can  be  measured  and 
denoted  as  Li  and  L2  respectively.  From  Fig.  3.1,  it  can  be  seen  that  the 
tangential  angle  of  the  curve  at  the  point  that  the  first  beam  strikes  the 
curve  can  be  measured  by 

0  =  arctan[(L2-Li)/b]  (3.1) 

The  surface  is  assumed  to  nearly  follow  the  tangent  vector 
between  digitized  points.  The  angle  calculated  from  (3.1)  is  the  amount 
that  the  LDS  orientation  will  change  for  the  next  digitizing  position. 
The  next  extrapolation  position  is  calculated  using  the  sampling  arc 
length,  s,  the  angle  0,  and  the  difference  between  Li  and  the  LDS  zero 
displacement  distance,  Lj.  It  should  be  noted  that  only  the 
measurement  made  by  the  first  laser  scanner  will  be  saved  as  a  surface 
measurement. 

The  distance  between  the  two  light  beams,  b,  is  determined  by 
the  maximum  curvature  of  the  surface  as  discussed  in  Chapter  4.  If  the 
maximum  curvature  is  large  on  a  surface,  b  must  be  small.  Otherwise, 
the  scanner  may  not  be  able  to  follow  the  surface.  Ideally,  b  is 
adjustable  to  accommodate  various  surfaces  and  precision 
requirements. 


The  concept  of  the  Multiple  Beam  Approach  using  three  LDS  can  be 
explained  by  the  illustration  of  Fig.  32. 
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Figure  3.2  Multiple  Beam  Approach  -  Three  LDS 


The  multiple  beam  approach  using  three  LDS  can  measure  the 
surface's  local  curvature.  The  three  LDS  are  mounted  so  that  the 
beams  are  parallel  to  each  other  like  in  the  two  beam  approach.  With 
the  additional  LDS,  three  points  can  be  measured  locally  on  the  surface. 
These  three  points  can  determine  local  curvature  by  calculating  the 
circle  which  will  pass  through  these  points.  This  circle  is  the  osculating 
circle  to  the  curve  and  has  the  same  curvature  as  the  curve  does.  The 
curve  is  assume  to  have  constant  curvature  between  digitized  points 


and  the  extrapolated  curve  follows  the  osculating  circle  as  shown  in 
Fig.  3.2. 

3.2  Extrapolation  Approach 

The  extrapolation  approach  uses  an  nth  order  polynomial  to 
extrapolate  the  sculptured  surface’s  height  and  orientation  at  the  next 
digitizing  location.  The  polynomial  has  the  standard  form: 

z(x)  =  anxn  +  an-ix0*1  +  ...  +  ajx  +  ao  (3.2) 

The  (n+1)  unknown  coefficients  in  the  polynomial  are  solved 
for  by  using  the  previous  (n+1)  measurements  of  the  sculptured 
surface.  For  example,  a  2nc*  order  polynomial  uses  the  last  three 
measured  surface  points  to  solve  for  the  unknown  coefficients  a2,  aj, 
and  ao-  Let  [(xi,yi,zi),  (x2,y2,Z2),  and  (x3,y3,Z3)]  represent  the  three  most 
recently  measured  points,  see  Fig.  3.3. 

If  digitizing  in  the  x  axis  direction,  then  y\,  yi,  and  y3  can  be  all 
equal.  In  addition,  each  point  must  satisfy  Eq.  (3.2).  Therefore,  from  Eq. 
(3.2),  the  three  constraining  equations  are: 

zi  =  a2X]2  +  aixi  +  ao 

Z2  =  32x22  +  alx2  +  30 
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z3  =  a2x32  +  alx3  +  ao 


Figure  3.3  2nd  Order  Polynomial  Extrapolation 

Once  the  coefficients  are  known,  Eq.  (3.2)  can  be  used  to 
extrapolate  the  surface’s  height  at  the  next  location  to  be  digitized. 
Surface  orientation  can  also  be  extrapolated  by  evaluating  the 
derivative  of  Eq.  (3.2),  see  Eq.  (3.3). 


z'(x)  =  (n)anXn_1  +  (n-l)an.ixn’2  +  ...  +  a\ 


(3.3) 
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Equation  (3.3)  gives  an  extrapolated  tangent  value  of  the 
sculptured  surface.  The  orientation  of  the  LDS  is  set  perpendicular  to 
this  tangent  value.  Thus,  the  laser  beam  is  oriented  so  that  its  slope  is 
set  equal  to  the  negative  inverse  of  z'.  The  z  axis  position  for  the  LDS 
is  calculated  by  Eq.  (3.2).  The  x  axis  value  for  the  position  was  used  in 
Equations  (3.2)  and  (3.3)  and  is  a  fixed  distance  apart  from  the  last 
measured  point.  This  distance  is  selected  by  the  operator  at  run  time. 

Unfortunately,  this  extrapolation  approach  needs  "n+1"  surface 
points  before  it  can  be  implemented.  For  n=2,  three  surface  points 
must  initially  be  measured  using  a  different  approach  before 
implementing  the  2nd  order  polynomial  extrapolation  approach.  One 
solution  is  to  use  the  polynomial  extrapolation  approach  but  limit  the 
extrapolation  order  to  the  highest  order  that  fits  the  data  collected. 
Therefore,  for  n=2,  a  Oth  order  polynomial  is  used  to  extrapolate  the 
2nd  point,  a  1st  order  polynomial  is  used  to  extrapolate  the  3rd  point, 
and  a  2nd  order  polynomial  is  used  to  extrapolate  the  remaining 
points.  Note,  a  Oth  order  polynomial  extrapolates  the  sculptured 
surface  as  a  plane  with  a  constant  z  axis  value. 

A  variation  to  the  approach  just  described  is  to  use  the  robots 
initial  orientation  when  extrapolating  the  2nd  point.  In  this  case,  the 
sculptured  surface  is  still  assumed  to  be  a  plane  but  the  plane  is 
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assumed  to  be  perpendicular  to  the  initial  orientation  of  the  robot. 
This  is  a  better  assumption  because  the  operator  would  naturally  place 
the  robot  in  a  near  perpendicular  orientation  before  starting  the 
digitizing  process.  This  variational  approach  was  used  in  the 
laboratory  to  extrapolate  the  2nd  point.  The  3rd  point  was  extrapolated 
using  a  1st  order  polynomial.  And  subsequent  points  were 
extrapolated  using  a  2nd  order  polynomial  as  described. 

Another  digitizing  location  that  needs  special  consideration,  is 
the  1st  point  on  subsequent  rows.  When  the  surface  has  been  digitized 
across  the  requested  x  length,  the  LDS  is  moved  in  the  y  direction  and 
the  digitizing  resumes  in  the  opposite  x  direction.  The  x  value,  z  value 
and  orientation  for  the  first  point  of  the  new  row  is  assumed  to  be  the 
same  as  the  last  point  of  the  preceding  row,  only  the  y  axis  value  is 
different.  Subsequent  points  on  the  new  row  are  extrapolated  as  they 
were  at  the  start  of  the  preceding  row.  Namely,  the  2nd  point  on  the 
row  is  extrapolated  assuming  the  sculptured  surface  is  a  plane 
perpendicular  to  the  LDS.  The  3rd  point  in  the  row  is  linearly 
extrapolated  by  using  the  1st  and  2nd  points  of  the  row.  All  subsequent 
points  in  the  row  are  extrapolated  using  a  2nd  order  polynomial. 

The  polynomial  extrapolation  equations  can  also  be 
prameterized  using  the  surface  arc  length  between  data  points  as  the 
parameter.  Using  this  approach  the  arc  length  between  data  points  is 
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held  constant  while  determining  what  the  next  digitizing  position 
should  be.  A  parameterized  variation  to  the  extrapolation  approaches 
is  used  in  Chapter  IV  when  developing  theoretical  limitations  of  these 
two  approaches.  However,  the  equations  develop  here  were  used  in 
the  experimental  implementation. 

3.3  Summary 

This  chapter  outlined  the  Multiple  Beam  Approach  and  the 
Polynomial  Extrapolation  Approach.  The  Multiple  Beam  Approach 
uses  a  set  of  LDS  positioned  parallel  to  each  other.  The  extrapolation 
curve  for  this  approach  is  a  constant  curvature  curve  (i.e.  a  circle  or  a 
line).  The  Polynomial  Extrapolation  Approach  uses  past  digitized 
points  to  evaluate  a  polynomial  which  passes  through  these  data 
points.  Note  that  if  the  polynomial  has  a  near  constant  curvature  at 
the  past  digitized  points  then  it  will  have  approximately  the  same 
curvature  at  the  next  digitized  point.  The  next  chapter  will  evaluate 
these  extrapolation  approaches. 


CHAPTER  IV 


EVALUATION  OF  EXTRAPOLATION  APPROACHES 

In  this  chapter,  the  digitizing  error  parameters  are  shown  to  be 
bounded  functions  when  using  the  proposed  prediction  approaches  to 
digitize  an  unknown  surface  which  has  a  finite  maximum  curvature. 
These  functions  can  be  used  to  determine  an  adequate  sampling  arc 
length,  if  the  maximum  surface  curvature  is  known,  or  for  a  given 
sampling  arc  length,  the  maximum  surface  curvature  that  can  be 
accurately  digitized  can  be  determined. 

This  chapter  will  develop  as  follows.  First,  a  general  overview 
of  the  digitizing  error  parameters  and  surface  curvature  is  presented. 
Next,  any  plane  curve  with  finite  maximum  curvature  is  shown  to  be 
bounded  locally  by  two  tangent  circles  of  maximum  curvature.  Given 
this  local  bounding  result,  the  maximum  and  minimum  possible 
slopes  of  the  curve  at  a  point  in  this  local  bounding  region  is 
developed.  Using  these  two  results,  the  worst  case  prediction  is  then 
established,  and  the  error  parameter  functions  are  derived  for  this  case. 
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4.1  General  Overview 

In  the  multiple  beam  approach  and  the  polynomial 

extrapolation  approach,  the  surface  is  digitized  one  row  at  a  time  where 
the  rows  are  formed  by  the  intersection  of  parallel  planes  with  the 
surface.  The  intersection  of  a  plane  with  the  surface  forms  a  plane 

curve.  Since  the  LDS’s  beam  is  kept  within  this  intersecting  plane 

while  digitizing  a  row,  the  objective  of  these  predicting  approaches  is  to 
extrapolate  the  plane  curve’s  position  and  normal  vector,  see  Fig.  3.2. 
In  this  evaluation  of  the  extrapolation  approaches,  the  plane  curve  is 
assumed  to  have  finite  maximum  curvature. 

There  are  three  error  parameters  associated  with  the  digitizing 
process:  p,  0,  and  As.  The  first  two  error  parameters,  p  and  0,  are  a 
result  of  limitations  associated  with  the  LDS.  p  is  the  displacement 
between  the  LDS  reference  position  and  the  object’s  surface.  0  is  the 
angle  between  the  laser  beam  and  the  surface  normal  vector,  see  Fig. 
4.1.  Both,  p  and  0  have  bounded  values  which  are  specified  by  the 
manufacturer.  If  p  is  too  large  then  the  LDS  will  be  out-of-range  and 
the  LDS  will  fail  to  give  a  displacement  reading.  If  0  is  too  large  then 
the  LDS  will  either  fail  or  give  inaccurate  readings.  The  last  error 
parameter.  As,  is  related  to  the  the  sampling  arc  length.  It  is  the 
difference  between  the  desired  sampling  arc  length,  s,  and  the  actual  arc 
length,  s  +  As,  as  measured  between  adjacent  digitized  points. 


40 


Minimizing  As  keeps  the  sampling  arc  length  uniform  which  is 
desirable  for  later  surface  reconstruction  from  the  digitized  points. 
This  chapter  shows  that  p,  0,  and  As  are  bounded  when  a  surface  with 
finite  maximum  curvature  is  digitized  using  one  of  the  two  proposed 
extrapolation  approaches. 


The  curvature,  k,  of  a  plane  curve  is  the  absolute  value  of  the 


rate  of  change  in  direction  of  the  unit  tangent  vector  at  a  point  P  on  the 

del 

curve  as  the  point  P  moves  along  the  curve,  k  = 


The  curvature 


can  also  be  written  as 
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j  d2w/du2  | 

K  =  [l+(dw/du)2p/2  '  (41) 

see  Fig.  4.2.  The  symbols  u  and  w  represent  orthogonal  unit  vectors  in 
the  plane  of  the  curve.  The  symbol  0  represents  the  angle  between  the 
tangent  vector  at  P  on  the  curve  and  u. 

A  circle  is  a  curve  which  has  constant  curvature,  and  its  radius, 
R,  is  related  to  the  curvature  by  R  =  1/k.  An  osculating  circle  or  circle 
of  curvature  is  a  circle  which  is  tangent  to  a  plane  curve,  lies  on  the 
concave  side  of  the  curve,  and  has  the  same  curvature  as  the  plane 
curve  at  the  point  of  intersection  [1}. 

4.2  A  Plane  Curve  With  Finite  Max.  Curvature  is  Bounded  Locally 

Theorem  4.1: 

A  plane  curve  which  is  known  to  have  a  finite  maximum 
curvature,  Km,  and  to  pass  through  the  point,  P0,  with  a  tangent 

vector,  T0,  is  bounded  locally  by  two  tangent  circles  with  radii,  R, 
where  R  =  1/Km,  see  Fig.  4.2.  A  sufficient  condition  for  locality  is 

I  s  I  <  ^  R  where  s  is  the  curve's  arc  length  measured  from  the 

point  P0.  The  curve  will  not  cross  into  these  tangent  circles  as 
long  as  the  arc  length  satisfies  this  condition. 
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Proof: 

Let  {u,w}  be  an  orthogonal  reference  frame  in  the  plane  of  the 
curve  with  its  origin  located  at  an  arbitrary  point  PQ/  and  its  principle 
axis,  u,  directed  along  the  tangent  vector,  T0,  see  Fig.  4.2. 


Figure  4.2  A  Plane  Curve  and  Bounding  Tangent  Circles 


Given  that 


k(s)  = 


d  8(s) 
ds 


the  curvature  is  bounded  so  that 
<*  Km,  the  angle  0(s)  is  bounded  by 


f  d  0(t)  f 

0(s)  =  J  ~&T  dx  -  J  Km'dT  =  %s- 

0  0 


(4.2) 


where  s  is  the  arc  length  of  the  curve  measured  from  PD. 
The  differential  positions  along  the  curve  are 
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du  =  cos{  0(s) }  ds, 
and 

dw  =  sin{  0(s) }  ds.  (4.3) 


Therefore,  positions  on  the  curve,  P(s)  =  (u(s),  w(s)}  are  calculated  as 


s 


u(s)  = 

1  cos{  0(x)  }  dx. 

0 

s 

(4.4) 

w(s)  = 

J  sin{  0(x) }  dx. 

0 

(4.5) 

Given  the  constraint  on 


d0(s) 
ds  ' 


corresponding  bounds  on  u(s) 


and  w(s)  can  be  determined.  u(s)  is  seen  to  be  maximized,  if  0(s)  =  0, 
which  gives  u(s)max  =  s.  u(s)  is  minimized  and  extreme  values  for  w(s) 

±71 

are  found  by  maximizing  0(s)  until  the  value  y  is  reached.  This  is 


d  0(s) 


done  by  holding  — constant  at  its  maximum  value  of  ±Km  for  I  s  I  < 


K 


j- — .  This  results  in  0(s)  =  ±Kms.  Using  this  value  for  0(s), 


v  1 

u(s)min  *  ^sin(Kms), 

(4.6) 

-1 

w(s)min  =  ^<l"COS(Kms)), 

(4.7a) 

and 
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w(s)maX  =  rr“(l-cos(Kms)). 


(4.7b) 


Thus,  given  an  arc  length  I  s  I  <  5— —  =  sm,  the  position,  P(s),  of 

the  curve  at  Si  is  bounded  by  one  of  two  rectangles  depending  on 
whether  S]  is  positive  or  negative.  These  rectangles  are  formed  by  the 
the  bounds  on  u(sj)  and  w(sj). 

For0<si  £2^r- 


— - sinUmS!)  <  ute^  £  si,  and 
*m 

“ -(l-COS(KmSi))  <,  w(S!)  <.  ~  (I-COS(KjjtjSj)). 


2^Ss><0' 


Sj  £  u(s|)  £  — sin(KmS!),  and 
~r“(l-cos(Kmsi))  <  w(sj)  <  (l-COS(Km  Sj)). 


(4.10) 


(4.11) 


A  graphical  representation  of  these  boundaries  when  s  =  Sj  is  shown  in 
Fig.  4.3. 
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Eliminating  S!  from  (4.8),  (4.9),  (4.10),  and  (4.11)  results  in  the 
following  equations 

u2  +  (w  +  1/Km)2  =  (1/Km)2, 

and 

u2  +  (w-l/Km)2  =  (l/Km)2  (4.12) 

These  are  the  equations  of  two  circles  with  radii,  R  =  1/Km,  which  are 
tangent  to  the  curve  at  the  point  P0.  Because  these  boundary  equations 
hold  for  all  values  of  I  s  I  £  sm,  the  curve  is  known  to  be  bounded  locally 
between  two  tangent  circles,  inclusively.  The  resulting  locally  bounded 


region  which  is  known  to  contain  the  plane  curve  for  I  s  I  £  sm  is  shown 
in  Fig.  4.4. 
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Thus,  a  plane  curve  with  maximum  curvature,  Km,  which 
passes  through  the  point  P0  and  has  a  tangent  vector  in  the  direction  of 
T0,  is  bounded  locally  by  two  tangent  circles  with  radii,  R,  where  R  = 

QED. 

As  a  corollary  to  Theorem  4.1,  the  sharpest  turn  that  a  plane 
curve  with  a  finite  maximum  curvature,  Km,  can  make  is  to  follow  an 
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osculating  circle  with  radius  R=l/Km.  This  result  will  be  used  to 
determine  the  worst  case  for  the  extrapolation  approaches. 

4.3  A  Curve's  Slope  Is  Bounded  In  The  Locally  Bounded  Region 

Suppose  the  following  four  things  are  known  about  a  plane 
curve:  1)  It  has  a  finite  maximum  curvature,  2)  It  passes  through  the 
point  PQ/  3)  It  has  a  known  tangent  vector,  T0,  at  the  point  PQ,  and  4)  It 
passes  through  the  point  P2  where  P2  is  located  in  the  locally  bounded 
region  of  Theorem  4.1.  With  these  constraints,  the  slope  of  the  curve 
at  the  point  P2  is  known  to  be  bounded,  see  Fig.  4.5.  Because  of 
symmetry,  only  the  case  where  u  >  0  will  be  considered.  The  path  that 
the  two  limiting  curves  take  will  now  be  described. 

Picture  that  you  are  on  top  of  the  curve  at  point  P0  and  are 
"driving"  it  in  the  positive  arc  length  direction  (i.e.  in  the  direction  of 
T0).  The  curve  with  the  largest  slope,  ml,  at  the  point  P2  starts  out  by 
following  a  constant  maximum  curvature  curve  to  the  right  (i.e.  the 
lower  boundary  circle)  and  then  switches  to  following  a  constant 
maximum  curvature  curve  to  the  left  to  reach  the  point  P2,  see  Fig.  4.5. 
The  curve  with  the  smallest  slope,  ms,  at  the  point  P2,  starts  out  by 
following  a  constant  maximum  curvature  curve  to  the  left  (i.e.  the  top 
boundary  circle)  and  then  switches  at  an  appropriate  place  to  following 
a  constant  maximum  curvature  curve  to  the  right  before  passing 
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through  the  point  P2.  Note,  that  the  values  of  ms  and  ml  are  functions 
of  the  location  of  P2.  However,  any  curve  which  satisfies  theses  four 
constraints  is  bounded  locally  (i.e.  between  the  point  P0  and  P2)  by  these 
two  extreme  curves  and  has  a  slope,  m,  at  the  point  P2  which  is 
bounded  by  ms  £  m  £  mj.  Also  note,  if  the  point  P2  lies  on  the  top 
boundary  circle,  then  ms  =  ml,  and  both  curves  simply  follow  the  top 
boundary  circle  from  the  start.  And  similarly,  if  the  point  P2  lies  on  the 
bottom  boundary  circle,  then  ms  =  ml,  and  both  curves  follow  the 
bottom  boundary  circle  to  get  to  the  point  P2. 


To  determine  the  worst  possible  orientation  that  the  plane  curve 
might  have  with  the  LDS's  beam,  maximum  and  minimum  slope 
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values  will  be  calculated  for  points  along  a  line  which  crosses  the 
locally  bounded  region.  This  line  represents  the  LDS’s  beam  and  it 
intersects  the  origin  of  the  lower  tangent  circle  while  forming  an  angle, 
y,  with  the  w  axis,  see  Fig.  4.6. 


This  would  be  a  reasonable  position  and  orientation  for  the 
LDS’s  beam,  if  the  curve  had  previously  (i.e.  for  s  £  0)  followed  this 
lower  circle. 

The  minimum  slope  and  maximum  slope  are  functions  of  y, 
and  AR/R  and  AR  is  the  distance  along  the  laser  beam  from 
intersection  with  the  lower  bounding  circle  to  the  point  P2.  Due  to 
geometry  constraints,  y  is  bounded  by 
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o<y<£ 


(4.13a) 


(i.e.  for  y  >  jr,  the  laser  line  will  not  intersect  the  top  circle).  The  ratio 
AR 

-jj-  is  bounded  by 


0  <  -p-  <  (2  cos(y)  -V4  cos2(y)-3-l). 


(4.13b) 


The  upper  bound,  R  (2  cos(y)  -V  4  cos2(y)-3  -1),  is  the  distance  along 

the  laser  beam  from  intersection  with  the  lower  bounding  circle  to  the 

upper  bounding  circle.  Because  the  curve  is  known  to  be  bounded  by 

AR 

these  two  circles,  this  distance  represents  the  largest  possible  value 
for  a  given  y  value. 

The  maximum  slope,  ml,  and  minimum  slope,  ms,  values  of  Fig.  4.5 
and  Fig.  4.6  can  be  determined  for  all  points  along  the  laser  beam  line 
between  these  two  bounding  circles.  Finding  the  maximum  values  for 
these  values  will  determine  the  worst  possible  orientation  the  LDS  will 
have  with  the  curve.  The  position  of  the  point  where  the  maximum 

r  AR  , 

and  minimum  slope  values  are  calculated  is  a  function  of  and  y. 
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Therefore,  the  maximum  slope  as  a  function  of  and  y  is 

found  by  first  solving  and  normalizing  the  triangle  in  Fig.  4.7  and  then 
finding  the  slope  of  the  curve.  The  max.  slope  function  is 

mi  =  tan(  B  +  C  -  y) 
where 

-  1  AR  -1  MR\2 1 

B  -  cos'l  <1  +  2  r  +  4  ^  R  J  y 

1  AR  1  fAR^l 
1+2R  +4^R  J  [ 

C  =  cos-1  - ^ - [  .  (4.14b) 

l  1+X  J 


(4.14) 

(4.14a) 


Figure  4.7 


Maximum  slope  Curve  Showing  Angles  {B,  C) 
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/>  rv 

The  minimum  slope  as  a  function  of  -jj-  and  y  is  found  by  first 

solving  and  normalizing  the  two  triangles  in  Fig.  4.8  and  then  finding 
the  slope  of  the  curve.  The  minimum  slope  function  is 


ms  =  tan(  A  -  D  -  F ) 


(4.15) 


where 


A  =  cos*l 


/x\ 2  AR  MR \2 

3  +  [rJ’2r  *[r  J 


(4.15a) 


D  =  cos'l 


w 


(4.15b) 


,  5  -lfx 

F  =  cos-l  4+4  r 


(4.15c) 


5  +  2 


AR  (A  R\2  ( 


AR>*  ,  % 
-4  -4  -p-  cos(  y) 


(4.15d) 


2R 


Fig.  4.8  Min.  Slope  Curve  Showing  Angles  {A,  D,  F}  and  Side  x. 


-0.2 


Maximum  slope  curves 


Minimum  slope  curves 


Figure  4.9  Extreme  Slope  Curves  as  a  function  of  with  LDS 

_  .  ^  .  ,1  n  3  n  1  it  5 

Orientation,  y,  Held  Constant.  From  left  to  right,  y  is  6'  g  6'  2  6'  8 

n  3  jt  7it  k 
6'  4  6'  8  6'  6  ' 
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The  plots  of  (4.14)  and  (4.15)  slope  for  seven  different  y  values  as 
AR 

a  function  of  are  shown  in  Fig.  4.9.  As  can  be  seen  from  these  plots, 

AR 

the  max/min  slope  functions  are  monotonically  increasing.  At  -p-  =  0, 

AR 

the  curve  is  perpendicular  to  the  LDS's  beam.  However,  as  -g- 

increases  the  inclination  angle  between  the  curve  and  the  laser  beams 
increases.  Thus,  the  worst  inclination  occurs,  (i.e.  0  is  maximized) 
when  the  curve  follows  the  top  tangent  circle.  The  other  error 
parameters,  p  and  As,  are  also  maximized  under  this  condition.  Fig.  4.9 
also  shows  that  the  maximum  errors  are  decreased  as  the  angle  y 
decreases.  Since  y  is  proportional  to  the  sampling  arc  length,  the 
maximum  errors  decrease  as  the  sampling  arc  length  is  decreased. 

4.4  Worst  Extrapolation  Case 

Both  the  multiple  beam  approach  using  three  LDS  and  the  2nd 
order  polynomial  approach  (which  uses  one  LDS)  have  the  same  worst 
case  performance  when  digitizing  a  surface  with  a  finite  maximum 
curvature.  These  two  approaches  are  comparable,  because  they  both 
approximate  the  surface's  local  curvature.  In  contrast,  a  multiple  beam 
approach  using  two  LDS  can  only  approximate  the  surface's  local 
tangent.  The  reason  both  methods  can  approximate  the  surface's  local 
curvature  is  that  both  methods  use  three  local  digitized  points. 
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As  discussed,  the  error  parameters  (0,  p,  and  As)  for  these 
approaches  are  found  to  be  monotonic  functions  of  the  sampling  arc 
length.  Because  the  leading  LDS  has  the  largest  sampling  arc  length 
out  of  the  three  LDS,  it  also  has  the  largest  error  parameters.  Both 
extrapolation  approaches  have  the  same  worst  case  error  parameters, 
because,  both  will  predict  the  same  worst  case  curve  to  follow.  This 
happens  if  the  leading  LDS  is  positioned  at  the  point  of  inflection 
where  the  curve  has  been  following  a  constant  positive  maximum 
curvature  and  then  switches  to  following  a  constant  negative 
maximum  curvature,  or  vice  versa.  This  situation  results  in  the 
maximum  error  between  the  predicted  position  and  orientation  for  the 
LDS  and  the  ideal  position  and  orientation  for  the  LDS,  see  Fig.  4.10. 


Figure  4.10 


Worst  Case  Curve  Estimation. 
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The  maximum  error  parameter  values  can  be  minimized  by 
using  a  linear  extrapolation  approach  (i.e.  a  two-laser  multiple  beam 
approach  or  a  1st  order  polynomial  approach).  In  this  approach,  the 
extrapolation  curve  would  be  a  line  parallel  to  T0  which  passes  through 
PG.  This  puts  the  extrapolation  curve  in  the  middle  of  the  locally 
bounded  region.  This  approach  would  be  the  best  if  there  was  rarely 
any  correlation  between  the  curve's  curvature  to  be  digitized  (the 
extrapolated  curvature)  and  the  curve's  curvature  previously 
measured.  More  commonly,  however,  the  curve's  curvature  is  near 
constant  over  several  consecutive  digitized  points.  Using  this  constant 
curvature  assumption,  the  average  error  parameter  values  would  be 
smaller  than  in  using  a  linear  approximation,  however,  the  maximum 
error  values  could  be  larger. 


4.5  Deriving  Error  Parameter  Functions 

Extreme  values  for  the  error  parameters  (0,  p,  and  As)  will  now 
be  calculated  as  functions  of  the  ratio  of  sampling  arc  length  to  radius 
of  maximum  curvature.  These  values  are  based  on  the  assumption 
that  the  plane  curve  being  digitize  has  a  bounded  maximum  curvature. 
As  discussed,  a  curve  with  finite  maximum  curvature  is  known  to  be 
bounded  locally  by  the  two  extreme  osculating  circles.  The  error 
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parameters  all  have  their  largest  values  if  the  estimated  curve  follows 
one  osculating  circle  but  the  actual  curve  follows  the  other  osculating 
circle. 


Because  of  symmetry,  the  maximum  error  values  are  the  same 
regardless  as  to  which  circle  is  the  estimated  curve  and  which  is  the 
actual  curve  or  which  side  the  curve  is  being  digitized  on.  For 
arguments  sake,  the  osculating  circle  on  the  right  is  assumed  to  be  the 
estimated  curve  and  the  osculating  circle  on  the  left  is  assumed  to  be 
the  actual  curve.  This  estimation  curve  is  reasonable  if  the  plane  curve 
had  been  following  a  constant  maximum  curvature  curve,  see  Fig.  4.10. 
This  estimation  curve  will  cause  the  LDS  to  be  positioned  and  oriented 
in  relation  to  the  actual  curve  so  that  the  error  parameters  are 
maximized. 

4.5.1  Relationship  Between  As/R  and  s/R 

The  error  parameter  As  is  the  difference  between  the  desired 
sampling  arc  length  and  the  actual  sampled  arc  length,  see  Fig.  4.11. 
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Figure  4.11  Measured  Curve’s  Arc  Length. 


Under  worst  case  conditions/  the  LDS  will  be  positioned  at  the 
point  Pe,  a  distance  s  along  the  lower  circle  from  the  last  measured 
point  Po,  and  will  be  oriented  so  that  the  laser  points  towards  the  center 
of  this  circle.  The  actual  point  measured,  Pm,  will  be  the  intersection 
of  this  "laser"  line  and  the  tangent  circle  on  the  left.  Using  the  law  of 
cosines,  the  law  of  sines,  and  trigonometric  identities,  the  relating 
equation  is 


where  0  < 


(4.16) 
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The  bound  on  p  is  the  result  of  the  problem's  geometry.  If  the  ratio  of 

arc  length  to  radius  is  greater  that  g-  then  the  LDS  might  not  measure 

the  actual  curve  at  all.  This  is  the  same  geometry  constraint  that 

As  s 

bounded  y  in  (4.13a).  A  plot  of  -p-  as  a  function  of  p  is  shown  in  Fig. 
4.12. 


s 

normalized  sampling  arc  length,  p 


4.5.2  Relationship  Between  p/R  and  s/R 


The  displacement  in  a  LDS  is  measured  from  the  reference  zero 
position  along  the  laser  beam's  projection  (line  of  sight  of  the  LDS)  to 
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the  object's  surface.  As  discussed,  the  reference  zero  position  is  placed 
on  the  estimated  curve.  Thus,  the  displacement,  p,  is  the  distance 
between  the  estimated  curve  and  the  actual  curve  as  measured  along 
the  LDS's  line  of  sight.  This  value  normalized  by  the  radius  of 
maximum  curvature  is  found  directly  from  using  the  law  of  cosines. 
The  equation  is 

R'2cos(l  )‘i"'\/(2cos(^!)'1)  <417> 

,  .  s  n 

where  0  <  r  <  g  • 


The  plot  of  (4.17)  is  shown  in  Fig.  4.13. 


Sampling  Arc  Length,  ^ . 
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4.5.3  Relationship  Between  6  and  s/R 

The  angle,  0,  is  the  angle  between  a  normal  vector  at  point  Pm  and  a 

vector  parallel  to  the  line  of  sight  of  the  LDS.  Thus,  the  cos(0)  is  equal 

to  the  inner  product  of  these  two  unit  vectors.  Using  this  result  and 

As 

substituting  the  right  hand  side  of  (4.16)  for  -|r,  the  relating  equation  is 


0  =  +  sin-1 1  sin 


¥)-sin(!)  V(2  cos(^}1)j  (4'18) 


,  -  s  7t 

where  0  < 


The  plot  of  0  as  a  function  of  is  shown  in  Fig.  4.14  and  a  composite 
plot  of  figures  4.12, 4.13,  and  4.14  is  shown  in  Fig.  4.15. 


Figure  4.14  Plot  of  Error  Parameter  0  as  a  Function  of  Sampling 
,  s 

Arc  Length,^- 
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4.6  An  Example  Of  Calculating  The  Sampling  Arc  Length 

The  error  parameter  plots  in  Fig.  4.15  can  be  used  to  select  the 
ratio  of  sampling  arc  length  to  radius  of  maximum  curvature,  As/R, 
that  will  bound  the  error  parameters  within  a  desired  range. 

For  example,  suppose  the  desired  bound  on  the  error  parameters 
is  as  follows:  101  <  12'  *  0.21  radians,  I  p  I  <10  mm,  and  I  As  I  <  0.05s. 
The  last  inequality  restricts  the  actual  sampled  arc  length  to  be  within 
5%  of  the  estimated  arc  length.  Suppose,  further  that  the  maximum 
radius  of  curvature  of  the  surface  is  R  =  4  mm.  As  seen  from  Fig.  10, 
for  the  error  parameter,  0,  to  be  bounded  so  that  1 0 1  <  0.21  rad,  then  the 
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s  p 

inequality  ^  <  0.1  must  be  satisfied.  Likewise,  for  ^  to  be  bounded  so 


that 


<  10/4  =  2.5,  then  the  inequality  r  <  5  must  be  satisfied  (this 


is  the  maximum  possible  value  ^).  And  finally,  for 

s^ 

R 


As 


R 


<  0.01^-  to  be 


S  5 

true  jj-  <  0.2  must  be  satisfied.  Consequently,  selecting  =  0.1  will 


satisfy  all  three  inequalities.  Therefore,  if  s  =  0.1  R  =  0.4  mm  is  used  as 
the  estimated  arc  length  value,  then  the  error  parameters  are  bounded 
as  follows:  101  <0.2  rad  =  11*,  Ipl  <  0.01R  =  0.04  mm,  and  I  As  I  < 
0.001  R  =  0.004  mm.  This  will  always  be  true  so  long  as  the  estimated 
curve  is  selected  so  that  its  maximum  curvature  less  than  or  equal  to 
the  maximum  curvature  of  the  digitized  curve  (i.e.  RS4  mm). 


The  error  parameter  plots  in  Fig.  4.15  can  also  be  used  to 
determine  the  limitations  on  the  surface  to  be  digitize  if  given  a 
particular  sampling  arc  length  and  LDS's  limitations.  Thus,  if  a 
sampling  arc  length  of  0.4mm  is  used  and  if  the  LDS  has  the 
limitations  that  101  <  12*  -  0.21  radians,  and  Ipl  <10  mm,  then  the 
surface  curvature  must  be  larger  than  l/(4mm)  to  be  sure  that  the 
measurement  readings  are  accurate. 
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4.7  Summary 

This  chapter  evaluated  how  well  the  extrapolation  approaches 
work  in  digitizing  an  unknown  surface  which  has  a  finite  maximum 
curvature.  The  error  parameters  (0,  p,  and  As)  were  defined.  They 
were  found  to  be  bounded  functions  of  the  ratio  of  sampling  arc  length 
to  radius  of  maximum  curvature.  These  functions  were  derived  by 
identifying  the  worst  prediction  case.  The  two  extrapolation 
approaches  were  shown  to  have  the  same  bounded  error  parameter 
functions.  The  worst  extrapolation  case  was  determined  by  using  the 
result  that  a  plane  curve  which  has  a  finite  maximum  curvature  is 
bounded  locally  by  two  tangent  circles  with  maximum  curvature.  The 
bounding  error  parameter  functions  can  be  used  to  determine  a 
sampling  arc  length  that  will  keep  the  error  parameters  within  desired 
bounds.  They  can  also  be  used  to  determine  the  maximum  surface 
curvature  which  can  be  accurately  digitized  given  a  particular  sampling 
arc  length  and  LDS  limitations. 


CHAPTER  V 


EXPERIMENTAL  SYSTEM 


This  chapter  presents  the  digitizing  system  which  was 
constructed  in  the  laboratory  to  demonstrate  and  evaluate  the  proposed 
extrapolation  approaches.  The  overall  requirements  and  architecture 
of  the  system  are  first  presented.  This  is  followed  by  a  discussion  of  the 
controlling  software.  Lastly,  the  nonlinear  position  accuracy  of  rotary 
joint  robots  is  illustrated  with  two  examples. 

5.1  System  Architecture 

The  digitizing  system  implemented  to  study  the  proposed 
extrapolation  approaches  needed  to  meet  several  requirements.  The 
system  had  to  be  capable  of  positioning  and  orienting  the  LDS  with  a 
reasonable  degree  of  accuracy.  The  implementation  of  the 
extrapolation  approaches  required  a  computation  and  a  control  unit 
that  could  accept  the  output  from  the  LDS,  control  the  re-positioning  of 
the  LDS,  and  record  collected  data.  Finally,  the  equipment  had  to  be 
chosen  from  what  was  on  hand  (where  possible)  due  to  budget 
constraints. 
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The  digitizing  system  that  was  constructed  consisted  of  two  Laser 
Displacement  Sensors,  a  six  degrees-of-freedom  PUMA  robot,  and  an 
IBM  PC,  see  Fig.  5.1. 


Figure  5.1  Major  Components  in  Digitizing  System 


A  fixture  mounted  on  the  robot's  end-effector  holds  the  two  LDS 
and  positions  their  beams  parallel  to  each  other  and  separated  by  a 
distance  of  17.85  mm.  The  analog  ±  5  volt  output  of  the  two  LDS  is 
converted  to  a  digital  value  by  a  MetraByte  DAS-8  board  in  the  IBM  PC. 
The  IBM  PC  communicates  to  the  PUMA  robot’s  controller  via  a  serial 
port.  As  needed,  the  control  software  on  the  IBM  PC  requests  the 
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robot's  current  position  and  directs  the  robot  to  move.  The  control 
software  uses  the  LDS  data  and  the  robots  position-orientation 
information  to  determine  data  points  on  the  sculptured  surface.  These 
data  points  are  then  used  to  implement  the  multiple  beam  approach  or 
the  polynomial  extrapolation  approach  described  above. 

Selected  specifications  on  the  major  hardware  components  are 
listed  in  Table  5.1. 


Table  5.1  Selected  Hardware  Specifications 

•  Keyence  LB-72/LB-12:  Laser  Displacement  Sensor 

•  Stand-off  distance:  40  mm 

•  Measurement  range:  ±10  mm 

•  Resolution:  ±  0.01 5  mm  (Object:  white  paper  at  a  distance  of 
40mm,  at  2ms) 

•  Linearity:  1%  of  the  displacement  (Object:  white  mat  paper, 

tilted  ±30°) 

•  PUMA  Mark  III  500  Series  Robot 

•  6  axes  of  rotation, 

•  914  mm  (3.0  ft)  reach  range, 

•  Position  Repeatability  ±  0.1  mm  (±  0.004  in) 

•  IBM  PC 

•  4.77  MHz  clock  speed 

•  MetraByte  DAS-8:  Data  Acquisition  Board 

•  ±  5  volt,  12  bit  Analog  to  Digital  Converter 
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5.2  System  Software 


The  control  software  was  made  up  of  several  low  level  and  high 
level  modules.  The  low  level  modules  include:  Serial 
communication.  Screen  memory  management,  and  data  acquisition. 
The  high  level  modules  include:  protocol  to  communicate  with  robot's 
controller,  terminal  emulation  mode,  digitizing  mode,  and  a  stand 
alone  user  program  running  on  the  robot's  controller,  see  Fig.  5.2. 


High  Laval  Modules  Low  Level  Modules  Hardware 


Figure  5.2  Control  Software  Modules 


The  low  level  serial  communication  software  was  constrained  to 
be  interrupt  driven  due  to  the  speed  at  which  the  robot's  controller 
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transmitted  serial  data.  The  interrupt  service  routine,  IRQ4,  written 
for  this  experiment,  uses  two  circular  queues  to  buffer  in-coming  and 
out-going  data.  When  called,  this  routine  checks  the  8250  UART  for 
receive-data  ready.  If  there  is  in-coming  data,  the  character  is  placed  in 
the  in-coming  queue.  Otherwise,  if  the  transmitter  holding  register  is 
empty  then  a  character  for  the  output  queue  is  transmitted.  The 
interrupt  service  routine  executes  fast  enough  to  prevent  an  overflow 
from  occurring.  Thus,  the  high  level  control  software  only  reads  from 
and  writes  to  the  in-coming  and  out-going  queues,  respectively,  to 
communicate  with  the  robot's  controller.  When  the  control  software 
wants  to  transmit  data  to  the  robot's  controller,  it  fills  the  output  queue 
with  data  and  calls  a  flush  routine  which  enables  interrupts  when  the 
UART's  transmitter-holding  register  is  empty.  After  the  out-going 
queue  is  empty,  the  flush  routine  disables  interrupts  on  transmitter¬ 
holding-register  empty  and  returns  to  the  calling  functions. 

The  screen  memory  management  routines  divide  up  the  screen 
memory  into  two  windows.  The  top  window  is  used  for  displaying 
terminal  emulation  mode  data.  The  bottom  window  is  used  for  the 
digitizing  control  mode.  Scrolling  is  supported  in  both  windows.  The 
center  line  of  the  screen  memory  is  used  for  displaying  current  LDS 
readings. 
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The  data  acquisition  module  is  used  to  read  the  LDS  voltage 
values  and  convert  them  to  displacement  units  (millimeters).  The 
DAS-S  board  uses  12  bits  for  the  analog-to-digital  conversion. 
Therefore,  the  displacement  measurements  are  quantized  in 
increments  of  0.005  mm  (20  mm  /  2A12  =  5  mm).  This  quantization 
value  is  5%  of  the  magnitude  of  the  robot’s  position  repeatability  and  is 
therefore  considered  negligible.  An  out-of-range  error  signal  from  the 
LDS  is  also  read  and  the  results  are  passed  on  to  higher  level  modules. 
If  an  out-of-range  error  is  detected,  then  the  higher  level  control 
modules  try  reorienting  the  LDS  in  order  to  get  a  good  reading. 

Higher  level  functions  are  written  to  handle  the  protocol  of  the 
robot's  controller.  These  functions  oversee  the  format  of  data 
transmitted  to  the  robot's  controller.  Because  the  robot's  controller 
operates  in  full-duplex,  all  data  transmitted  to  the  robot  is  checked  for 
transmission  errors.  If  a  transmission  error  is  detected,  an  abort  signal 
(AU)  is  sent  to  the  robot's  controller  and  the  command  line  is  re¬ 
transmitted. 

This  software  first  runs  in  terminal  emulation  mode.  In  this 
mode,  the  robot  is  operated  as  if  it  is  connected  to  a  terminal  and  the 
robot  can  be  manually  positioned  into  its  starting  position  before 
digitizing  begins. 
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The  control  mode  modules  coordinates  and  controls  the 
digitizing  operation.  This  module  reads  the  LDS  values  and  the  robot's 
position  and  orientation.  It  implements  the  surface-to-sensor 
orientation  approaches  and  saves  all  digitized  surface  data  points  in  a 
file.  The  control  mode  is  initiated  from  the  terminal  mode  by  using 
the  hot  key,  (AK).  Once  initiated,  the  software  first  prompts  the  user  for 
overall  control  information,  such  as:  output  data  file  name,  size  of  area 
to  digitize,  number  of  digitizing  points  to  collect,  and  orientation 
approach  to  use. 

A  small  VAL-II  user  program  is  executed  on  the  robot's 
controller  concurrently  with  the  control  software.  This  program  is 
menu  driven  and  has  two  options.  Option  1  outputs  the  robot’s 
current  position  and  option  2  reads  (X,Y,Z,0,A,T)  data  to  reposition  the 
robot.  The  program  uses  prompts  to  identify  which  mode  it  is  in.  The 
"?"  prompt  signifies  the  program  is  waiting  for  an  option  selection. 
The  program  displays  a  ">"  prompt  when  it  is  waiting  for  (X,Y,Z,0,A,T) 
data  and  it  uses  the  prompt  to  signify  that  the  robot  has  not 
completed  a  move. 

5.3  Nonlinear  Robot  Position  Accuracy 

The  PUMA  robot  used  in  this  experiment  has  six  rotary  joints. 
With  six  degrees-of-freedom,  the  robot  was  able  to  position  the  LDS  in 
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any  desired  orientation.  However,  rotary  joints  have  a  nonlinear 
relation  between  changes  in  joint  angles  and  changes  in  position  of  the 
end-effector.  Therefore,  the  accuracy  in  positioning  the  end-effector  is  a 
function  of  the  robot’s  configuration.  In  comparison,  machining 
centers  and  coordinate  measuring  machines  use  three  screw  (or 
prismatic)  joints  which  are  orthogonal  to  each  other,  and  changes  in 
the  screw  joint  angles  are  linearly  related  to  changes  in  the  tool’s 
position.  This  linear  relation  holds  for  all  tool  positions  in  the 
operating  envelope.  These  linear-orthogonal  joints  are  able  to  achieve 
higher  positioning  accuracy  over  rotary  joints.  But  rotary  joints  are 
necessary  to  dynamically  reorient  the  LDS.  This  section  will  illustrate 
the  nonlinearities  of  rotary  joints  using  a  two  link  robot. 


Figure  5.3  Two  link  Robot 


The  nonlinearities  of  positioning  a  robot  with  rotary  joints  will 
be  illustrated  using  the  two-link  robot  arm  shown  in  Fig.  5.3.  The  joint 
angles  are  0i  and  02  are  used  to  position  the  two  links  of  length  ai  and 
a2-  The  position  of  the  end-effector  as  a  function  of  joint  angles  is 
given  by 

P=[X/y]T  =  f(0)  5.1 

where  P  €  R2,  T  represents  transpose,  0  =  [  0i,  02  ]T  €  R2, 

x  =  ai  cos(0i)  -  a2  cos(0i+02), 
y  =  ai  sin(0i)  -  a2  sin(0i+02), 

0^  0i  <i  2n,  and  0  £  02 

First,  let  the  joint  angles  be  quantized  so  that  0i  =  m  A0  and  02  = 
n  A0  where  m  =  0,1,2,3,...  and  n  =  0,1,2,3,...  (This  will  typically  be  the 
case  if  optical  encoders  are  used).  To  give  a  simple  graphical 
illustration  of  the  robot's  nonlinearities,  the  operating  envelope  of  the 
two-link  robot  of  Fig.  5.3  is  shown  in  Fig.  5.4  where  A0  =  Jt/8  and  al=2 
a2-  It  is  also  shown  in  Fig.  5.5  where  A0  =  n/8  and  al=a2-  The  semi¬ 
circles  represents  the  path  of  the  end-effector  as  0i  is  held  constant  at  m 
A0  and  02  varies  from  0  to  n.  The  concentric  circles  in  the  figures 
represents  the  path  that  the  end-effector  would  take  if  02  is  held 
constant  at  n  A0  and  0]  varies  from  0  to  2tc.  The  intersection  of  the 
circles  and  semicircles  are  all  possible  end-effector  positions  due  to  the 
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quantization  of  the  joint  angles.  The  density  of  these  intersection 
points  represents  the  positioning  resolution  of  the  robots.  Higher 
position  accuracy  corresponds  to  the  point  being  closer  together.  Thus, 
the  highest  accuracy  in  the  x  axis  is  obtain  when  02  =  it,  and  the  highest 
accuracy  in  the  y  axis  is  obtain  when  02  =  0. 


Figure  5.4  Positioning  Locations  within  the  Operating 
Envelope  of  a  Two-Link  Robot  Arm  with  ai=2  a2- 


Figure  5.5  Positioning  Locations  within  the  Operating 
Envelope  of  a  Two-Link  Robot  Arm  with  ai  =  &2 

Now,  let  0i  and  02  take  on  all  real  values  within  their  respective 
ranges.  Then,  the  time  derivative  of  (5.1)  is 

dP  ppapid© 
dt  ~  _50i  502.  dt  ' 
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where 
9P  9P‘ 

90^  00^  =  J(  ©  )  e  R2x2  which  is  called  the  Jacobian, 

dP 

0*0 

=  [-ai  sin (0i )  +  &2  sin(0i+02),  ai  cos(0i)  -  a2  cos(0i+02)]t, 

and 

ap 

^  =  [a2  sin'**-  -02),  -a2  cos(0i+02>]t. 

Thus,  J(  0  )  relates  the  change  in  joint  angles  to  the  change  in 

end-effector  position.  In  [49,50],  Yoshikawa  used  the  Jacobian  of  a  robot 

to  define  the  manipulability  ellipsoid.  He  showed  that  the  subset  Sy  of 

dP  d  0  d0 

***  0*0  0*0 

in  the  space  Rm  realizing  such  that  — <  1  is  an  ellipsoid 

where  the  principal  axes  are  found  directly  from  the  singular  value 
decomposition  of  the  Jacobian.  He  used  the  ellipsoid  to  introduce  the 
idea  of  manipulability,  however,  these  ellipsoids  can  also  be  used  to 
determine  a  robot's  position  accuracy  for  a  given  configuration. 

The  ellipsoids  map  incremental  changes  in  the  position  of  the 
end-effector  for  a  given  set  incremental  changes  in  the  rotary  joint 
angles.  As  seen  in  the  previous  example  where  the  rotary  joint  angle  is 
measured  using  an  optical  encoder,  smaller  changes  in  the  end- 
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effectors  position  for  the  same  change  in  the  rotary  angle  corresponds 
to  higher  positioning  accuracy.  The  two-link  robot  with  aj=2a2  was 
seen  to  have  the  highest  resolution  with  02  *  0.  A  small  ellipsoid 
represents  small  end-effector  movement  for  the  given  set  of  joint 
angular  motion.  Thus,  smaller  ellipsoids  correspond  to  higher 
positioning  resolution. 


Figure  5.6  Manipulability  Ellipsoids  with  ai  =  2  a2. 
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dP 

The  principal  axes  of  the  ellipsoid  shows  the  direction  in  which  — 

will  have  the  highest  and  lowest  resolution.  If  the  axes  have  the  same 
magnitude  then  the  resolution  will  be  equal  in  all  directions.  These 
ellipsoids  for  the  two-link  robot  have  been  plotted  in  Fig.  5.6  and  Fig. 
5.7  for  select  robot  configurations. 


Figure  5.7  Manipulability  Ellipsoids  with  ai  =  a2- 
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Fig.  5.6  shows  the  robot  with  a\  =2  a2,  0i  =  tc/4,  and  02  =  n  it/ 8 
where  n=0,l,2,..,8-  Fig.  5.7  shows  the  robot  with  ai=a2  and  the  end- 
effector  moving  radially  out  at  it/ 4.  Note  that  the  smellier  ellipsoids  are 
located  in  the  same  area  as  the  higher  resolution  areas  of  Fig.  5.4  and 
Fig.  5.5.  These  examples  illustrate  that  rotary  joint  robots  have 
nonlinear  positioning  accuracy.  That  is  the  accuracy  of  the  robot 
changes  for  different  configurations.  Evaluating  the  manipulability 
ellipsoids  over  the  operation  envelope  of  the  robot  identifies  areas 
were  the  robot  has  the  highest  positioning  accuracy. 

5.4  Summary 

This  chapter  outlined  the  system  architecture  of  the  digitizing 
system  designed  for  this  research.  Both  the  hardware  and  the  software 
of  the  system  was  explained.  The  nonlinear  positioning  accuracy  of  an 
ideal  robot  with  rotary  joints  was  illustrated  using  a  two  link  robot. 
The  next  chapter  will  discuss  the  experiments  conducted  with  this 
digitizing  system. 


CHAPTER  VI 


EXPERIMENT 


This  chapter  discusses  the  procedures  used  to  calibrate  the 
digitizing  system.  A  mesh  diagram  of  the  data  collected  from  digitizing 
a  simple  concave  and  convex  surface  is  also  presented.  The  digitizing 
rate  of  the  system  was  measured.  A  qualitative  analysis  of  the 
performance  of  the  two  extrapolation  methods  is  given,  and  an 
observation  that  high  surface  reflectivity  adversely  affects  the  digitizing 
system  is  reported. 

6.1  Calibration 

Before  using  the  digitizing  system,  the  Keyence  LB-72/LB-12 
Laser  Displacement  Sensors  were  calibrated  and  a  transformation 
matrix  from  the  end-effector  to  the  LDS  was  defined.  The  LDS  have 
two  calibration  adjustments.  One  is  the  0-ADJ  adjuster  trimmer  which 
adjusts  the  0  volt  stand-off  distance.  The  stand-off  distance  on  one  LDS 
was  set  to  roughly  the  center  of  its  measuring  range.  Then,  the  other 
LDS  was  adjusted  carefully  to  have  the  same  stand-off  distance  as  the 
first  LDS.  The  second  calibration  adjustment  is  the  SPAN  adjuster 


trimmer.  This  adjustment  varies  the  volts  per  displacement  value. 
Both  LDS  were  calibrated  to  5  volts  per  10  mm  displacement. 
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After  the  lasers  were  calibrated,  the  transformation  matrix  from 
the  end-effector  to  the  0  volt  reference  position  was  determined.  The 
PUMA  robot's  controller  has  a  reserved  name  for  this  transformation 

* 

matrix,  named  "TOOL".  Once  this  transformation  matrix  is  defined, 

the  robot  can  be  positioned  using  locations  for  the  LDS  rather  than  the 

end-effector.  Robotic  system  calibration  has  been  studied  previously, 
i 

but  not  in  the  context  of  the  robot  being  used  as  a  measurement  device 
[45,46,47]. 


) 

63  Digitizing  Rate 


► 


The  overall  digitizing  rate  was  measured  in  the  laboratory.  The 
results  are  shown  in  Table  6.1.  The  rate  was  measured  while  using  the 
Multiple  Beam  Approach  to  digitize  a  plane. 


► 


i 


Table  6.1  Digitizing  Rate 


Motor  Saeed  :  Sampling  Distance  :  Digitizing  Rate 

5%  -  10  mm 

-  0.77  (data  points  /  second) 

5%  -  1  mm 

-  0.91 

100%  -  10  mm 

-  1.00 

100%  -  1  mm 

-  0.91 

> 


I 
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The  serial  communication  and  error  checking  between  the  IBM 
PC  and  the  robot’s  controller  is  believed  to  contribute  significantly  to 
the  time  needed  to  digitize.  Speed  improvements  could  possibly  be 
made  by  using  a  more  sophisticated  communication  protocol  available 
on  the  robot's  controller.  The  serial  communication  interface  used  was 
chosen  because  it  was  thought  to  be  faster  to  implement.  In  addition, 
the  PUMA  robot  controller  uses  a  deceleration  mode  when  the  arm 
approaches  the  final  position.  If  the  overall  distance  to  go  is  small,  the 
arm  does  not  reach  its  full  speed  but  always  remains  in  the  deceleration 
mode.  Thus,  the  communication  protocol  used  and  the  deceleration 
mode  is  believed  to  be  the  major  reason  the  digitizing  speed  is  not 
significantly  affected  by  the  motor  speed  setting. 

6.3  Demonstration  of  Extrapolation  Approaches 

Both  the  Multiple  Beam  Approach  and  the  Extrapolation  Approach  to 
maintaining  surface-to-sensor  orientation  were  demonstrated  by 
digitizing  a  complex  surface.  The  complex  surface  used  is  a  draw  die 
model  of  an  automoble  fire  wall,  see  Fig.  6.1.  The  model  was  donated 
courtesy  of  General  Motors.  The  left  half  of  the  model  was  digitized 
and  ploted  in  Fig.  6.2.  A  sampling  arc  length  of  2.5  mm  was  used. 

Regardless  of  the  approach  used,  the  surface  is  digitized  in  a  grid 
pattern.  The  distance  between  adjacent  points,  as  measured  along  the 


Figure  6.1 


Photograph  of  Complex  Surface  and  Digitizing  System 
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Figure  6.2  Plot  of  Digitized  Complex  Surface  (units=mm) 


coordinate  axis,  is  the  same  in  any  row  or  in  any  column.  The  surface 
is  digitized  row  by  row,  where  a  row  is  parallel  to  the  x  axis.  When  the 
end  of  the  row  is  reached,  digitizing  continues  on  the  next  row  but  in 
the  opposite  direction.  Therefore,  if  a  row  is  digitized  in  the  positive  x 
direction,  then  the  next  row  will  be  digitized  in  the  negative  x 
direction.  For  the  Multiple  Beam  Approach,  the  second  laser  would  be 
leading  the  digitizing  laser  on  one  row  and  following  the  digitizing 
laser  on  the  next  row. 

The  Multiple  Beam  Approach  using  two  LDS  gave  consistent, 
reliable  results  provided  the  surface's  radius  of  curvature  was  large  in 
comparison  with  the  distance  between  the  two  lasers.  Smaller  radius 
of  curvatures  produce  larger  errors  between  the  predicted  and  the 
actual  position  and  orientation  of  the  sculptured  surface,  as  expected. 
This  observation  is,  to  some  degree,  intuitive  if  you  consider  digitizing 
the  cross-section  of  a  semi-circle  which  has  a  smaller  diameter  than  the 
distance  between  the  two  laser  beams.  In  this  extreme  case,  only  one 
laser  beam,  at  most,  shines  on  the  semi-circle  at  any  time.  Under  this 
condition,  the  Multiple  Beam  Approach  fails.  However,  the 
Extrapolation  Approach  could  be  used  because  the  distance  between 
measured  points  can  be  adjusted  as  discussed  in  chapter  IV. 

For  the  polynomial  extrapolation  approach,  a  2nd  order 
polynomial  extrapolation  method  was  implemented.  This  order  of 


polynomial  was  observed  to  be  sensitive  to  errors  in  surface  point 
measurements.  This  sensitivity  is  increased  with  smaller  digitizing 
intervals.  This  is  illustrated  in  Fig.  6.3. 


Figure  6.3  2nd  Order  Polynomial  Extrapolation  with  Bad  Data  Point 


Note  that  changes  in  the  middle  point  changes  the  predicted 
surface  from  concave  to  convex.  In  this  case,  the  distance  between 
digitized  points  in  the  X  axis  was  held  constant  rather  than  the  arc 
length  as  in  chapter  IV. 


6.4  Surface  Reflectivity 


The  reflectivity  of  the  sculptured  surface  affected  the  LDS’s 
ability  to  function  properly.  The  LDS  failed  to  provide  correct 
displacement  readings  when  digitizing  highly  reflective  surfaces.  If  a 
surface  is  highly  reflective  and  the  LDS  is  oriented  so  that  the  laser 
beam  is  reflected  away  from  the  sensor  in  the  LDS  then  an  error 
reading  will  result. 

6.5  Angle  of  Incidence  Error 

The  measurement  error  caused  by  the  LDS  being  oriented  in  a 
non-perpendicular  (90*)  orientation  with  the  surface  was  evaluated. 
This  was  done  by  repetitively  digitizing  the  same  points  on  a  flat 
surface  with  the  LDS  oriented  at  60*,  90*,  and  120*.  The  surface 
digitized  was  a  sheet  of  paper  taped  onto  a  plane  of  glass.  For  each  LDS 
orientation,  thirty  points  parallel  to  the  x  axis  were  digitized  five  times 
and  the  average  z  axis  values  determined  The  largest  variation 
between  the  z  axis  readings  was  <±0.005"  which  is  within  the 
repeatability  of  the  robot  and  accuracy  of  the  LDS.  The  difference 
between  the  average  values  with  the  LDS  at  60*  and  the  average  values 
with  the  LDS  at  90*  is  plotted  in  Fig.  6.4.  Also,  the  difference  between 
the  average  values  with  the  LDS  at  120*  and  the  average  values  with 
the  LDS  at  90*  is  plotted  in  Fig.  6.5. 
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The  average  z  axis  values  with  the  LDS  oriented  at  90*  has  the 
smallest  LDS  measurement  error.  The  difference  between  this  value 
and  the  average  values  measured  with  the  LDS  oriented  at  60*  and 
120*  shows  the  measurement  errors  caused  by  a  non-perpendicular 
orientation.  These  errors  are  a  combination  of  the  inaccuracy  in  the 
LDS  measurement  and  in  the  inaccuracy  in  calibrating  the  robot.  The 
average  error  value  of  the  plot  in  Fig.  6.4  is  -1.50  mm  and  the  average 
error  value  of  the  plot  in  Fig.  6.5  is  1.43  mm.  These  are  significant 
errors  and  they  illustrate  the  importance  of  maintaining  a 
perpendicular  orientation  between  the  LDS  and  the  surface. 


Figure  6.4  The  Difference  Between  the  Average  Digitized  z  Axis 
Value  with  the  LDS  Oriented  at  60’  and  the  Average  Value  with 
the  LDS  Oriented  at  90* 
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Figure  6.5  The  Difference  Between  the  Average  Digitized  z  Axis 
Value  with  the  LDS  Oriented  at  120*  and  the  Average  Value  with 
the  LDS  Oriented  at  90* 


6.6  Summary 


This  experiment  demonstrated  the  feasibility  of  both 
extrapolation  methods  being  implemented  in  a  digitizing  system.  They 
were  successful  at  keeping  the  LDS  within  desired  position  and 
orientation  limits  while  digitizing  simple  concave  and  convex  surfaces. 
In  addition,  the  experiment  provided  some  insight  of  the  approaches 
sensitivity  to  bad  data  and  of  the  LDS  sensitivity  to  surface  reflectivity. 


CHAPTER  VII 


CONCLUSIONS 


This  Master's  research  project  implemented  a  robotic  non- 
contact  3-D  digitizing  system  to  digitize  unknown  sculptured  surfaces. 
Two  extrapolation  approaches,  the  multiple  beam  approach  and  the 
polynomial  extrapolation  approach,  were  demonstrated  with  this 
digitizing  system.  This  project  illustrated  that  more  general  sculptured 
surfaces  can  be  digitized  using  a  six  degrees-of-freedom  robot  to 
position  and  orient  the  LDS  than  can  be  digitized  using  a  three  axis 
coordinate  measuring  machine  or  machining  center. 

Another  advantage  of  this  digitizing  system  is  that  it  was  built 
on  feasibly  available  and  relatively  low  cost  equipment,  such  as,  robot 
manipulators  and  a  microcomputers. 

As  discussed  in  chapter  II,  keeping  the  laser  beam  perpendicular 
to  the  surface  is  a  key  factor  that  affects  the  precision  and  robustness  of 
the  digitizing  process.  This  research  has  shown  that  if  the  surface  has  a 
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finite  maximum  curvature,  then  a  sampling  arc  length  can  be  selected 
which  will  keep  the  error  parameters  of  the  LDS  within  desired 
bounds.  The  positioning  accuracy  of  a  rotary  joint  machine  was  show 
to  be  nonlinear.  The  positioning  accuracy  of  the  robot  is  a  limiting 
factor  in  the  overall  accuracy  of  this  digitizing  system. 

Typically,  digitizing  a  sculpture  surface  produces  more  data 
points  than  required  to  describe  the  surface.  In  this  case,  the  data  points 
can  be  lumped  together  to  create  a  mathematical  model  to  describe  the 
surface.  The  mathematical  surface  models  provides  a  more  flexible 
and  efficient  way  of  describing  the  surface.  More  work,  however,  needs 
to  be  done  to  automate  the  process  of  going  from  digitized  data  points 
to  a  mathematical  surface  model. 


APPENDIX 


COMPANY  REFERENCE  LIST 


Laser  Displacement  Sensors 

Aromat  Corporation 
629  Central  Avenue 
New  Providence,  NJ  07974 
Phone:  800-228-2350 

CyberOptics  Corporation 
2331  University  Avenue  SE 
Minneapolis,  Minnesota  55414 
Phone:  612-331-5702 

Keyence  Corporation  of  America 
17-17  Route  208  North 
Far  Lawn,  NJ  07410 
Phone:  800-328-2238 

Kohgakusha  Engineering  Co.,  LTD 
Osumi-Bld.,  Higashi-Dcebukuro  2-39-2 
Toshima-Ku,  Tokyo,  Japan  T170 
Phone:  03-987-6331 

Micro  Switch 

Honeywell  Inc. 

11  West  Spring  Street 
Freeport,  Illinois  61032-9945 
Phone:  815-235-6600 
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Perceptron 
23855  Research  Drive 
Farmington  Hills,  Michigan  48024 
Phone:  313-478-7710 

Renishaw  Inc. 

623  Cooper  Court 
Schaumburg,  IL  60173 
Phone:  708-843-3666 

Selcom 

P.O.  Box  250 

Valdes,  North  Carolina  28670 
Phone:  704-874-4102 

SAM,  Inc. 

6360  Hawthorne  Drive 
Winsor,  Ontario  (Canada)  N8T1J9 
Phone:  519-944-6641 

M  edar 

38700  Grand  River  Avenue 
Farmington  Hills,  Michigan  48335 
Phone:  313-477-3900 


Laser  Digitizing  Systems 

Chesapeake  Laser  Systems 

4473  Forbes  Blvd. 

Lanham,  Maryland  20706 
Phone:  301-459-7977 

Hymark  Ltd. 

Ottawa,  Canada 
Phone:  613-727-1584 

Laser  Design  Inc. 

9401  James  Avenue  South  •  Suite  162 
Minneapolis,  Minnesota  55431 
Phone:  612-884-9648 

Okada  Machinery  Corporation 

c/o  Tomen  America  Inc, 

Chicago  Branch 
1000  Corporate  Grove  Drive 
Buffalo  Grove,  IL  60089-4507 
Phone:  708-215-6850 

Robotic  Vision  Systems  Inc.  (RVSI) 

425  Rabro  East 

Hauppauge,  New  York  11788 

Sharnoa  Corporation 
23996  Freeway  Park  Drive 
Farmington  Hills,  Michigan  48335 
Phone:  313-476-2600 
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